R/real time software.txt 26

I also got onto the server and dumped the live weather station datas table, its contents are in the attached file along with the line from the realtime. It looks like things are attributed to the wrong fields. Note that the server time is out by 1 hour thinks its in GMT. Any help would be gratefully received. Latest Post: New Weather. Forum Icons: Forum contains no unread posts Forum contains unread posts Mark all read.

Lost your password? Please enter your email address. You will receive mail with link to set new password. Community Support. Forums Members Recent Search Login. Forums Members Recent Search. English Support. General Questions.

Last Post. Active Member. The file is generated by WeatherCat 3. Pierre Lannoy. Member Admin. Reply Quote. Thanks for the quick response! Better than from paid plug-ins! Well, that will cause a problem won't it! The R base function read. The data will be imported as a data frame. Note that, depending on the format of your file, several variants of read. To know your current working directory, type the function getwd in R console. If you use the R code above in RStudio, you will be asked to choose a file.

If your data contains column with text, R may assume that columns as a factors or grouping variables e. To read the file into R, ignoring the first three lines, we type:. To store the data in a time series object, we use the ts function in R. Sometimes the time series data set that you have may have been collected at regular intervals that were less than one year, for example, monthly or quarterly.

An example is a data set of the number of births per month in New York city, from January to December originally collected by Newton. We can read the data into R by typing:.

Once you have read a time series into R, the next step is usually to make a plot of the time series data, which you can do with the plot. For example, to plot the time series of the age of death of 42 successive kings of England, we type:.

We can see from the time plot that this time series could probably be described using an additive model, since the random fluctuations in the data are roughly constant in size over time.

We can see from this time series that there seems to be seasonal variation in the number of births per month: there is a peak every summer, and a trough every winter.

Again, it seems that this time series could probably be described using an additive model, as the seasonal fluctuations are roughly constant in size over time and do not seem to depend on the level of the time series, and the random fluctuations also seem to be roughly constant in size over time. Similarly, to plot the time series of the monthly sales for the souvenir shop at a beach resort town in Queensland, Australia, we type:.

In this case, it appears that an additive model is not appropriate for describing this time series, since the size of the seasonal fluctuations and random fluctuations seem to increase with the level of the time series. Thus, we may need to transform the time series in order to get a transformed time series that can be described using an additive model. For example, we can transform the time series by calculating the natural log of the original data:. Here we can see that the size of the seasonal fluctuations and random fluctuations in the log-transformed time series seem to be roughly constant over time, and do not depend on the level of the time series.

Thus, the log-transformed time series can probably be described using an additive model. Decomposing a time series means separating it into its constituent components, which are usually a trend component and an irregular component, and if it is a seasonal time series, a seasonal component.

A non-seasonal time series consists of a trend component and an irregular component. Decomposing the time series involves trying to separate the time series into these components, that is, estimating the the trend component and the irregular component.

To estimate the trend component of a non-seasonal time series that can be described using an additive model, it is common to use a smoothing method, such as calculating the simple moving average of the time series. For example, as discussed above, the time series of the age of death of 42 successive kings of England appears is non-seasonal, and can probably be described using an additive model, since the random fluctuations in the data are roughly constant in size over time:.

Thus, we can try to estimate the trend component of this time series by smoothing using a simple moving average. To smooth the time series using a simple moving average of order 3, and plot the smoothed time series data, we type:.

There still appears to be quite a lot of random fluctuations in the time series smoothed using a simple moving average of order 3. Thus, to estimate the trend component more accurately, we might want to try smoothing the data with a simple moving average of a higher order.

This takes a little bit of trial-and-error, to find the right amount of smoothing. For example, we can try using a simple moving average of order The data smoothed with a simple moving average of order 8 gives a clearer picture of the trend component, and we can see that the age of death of the English kings seems to have decreased from about 55 years old to about 38 years old during the reign of the first 20 kings, and then increased after that to about 73 years old by the end of the reign of the 40th king in the time series.

A seasonal time series consists of a trend component, a seasonal component and an irregular component. Decomposing the time series means separating the time series into these three components: that is, estimating these three components.

This function estimates the trend, seasonal, and irregular components of a time series that can be described using an additive model. For example, as discussed above, the time series of the number of births per month in New York city is seasonal with a peak every summer and trough every winter, and can probably be described using an additive model since the seasonal and random fluctuations seem to be roughly constant in size over time:.

For example, we can print out the estimated values of the seasonal component by typing:. The estimated seasonal factors are given for the months January-December, and are the same for each year. The largest seasonal factor is for July about 1. The plot above shows the original time series top , the estimated trend component second from top , the estimated seasonal component third from top , and the estimated irregular component bottom.

We see that the estimated trend component shows a small decrease from about 24 in to about 22 in , followed by a steady increase from then on to about 27 in If you have a seasonal time series that can be described using an additive model, you can seasonally adjust the time series by estimating the seasonal component, and subtracting the estimated seasonal component from the original time series.

You can see that the seasonal variation has been removed from the seasonally adjusted time series. The seasonally adjusted time series now just contains the trend component and an irregular component.

If you have a time series that can be described using an additive model with constant level and no seasonality, you can use simple exponential smoothing to make short-term forecasts. The simple exponential smoothing method provides a way of estimating the level at the current time point. Smoothing is controlled by the parameter alpha; for the estimate of the level at the current time point.

The value of alpha; lies between 0 and 1. Values of alpha that are close to 0 mean that little weight is placed on the most recent observations when making forecasts of future values. We can read the data into R and plot it by typing:. You can see from the plot that there is roughly constant level the mean stays constant at about 25 inches. The random fluctuations in the time series seem to be roughly constant in size over time, so it is probably appropriate to describe the data using an additive model.

Thus, we can make forecasts using simple exponential smoothing. For example, to use simple exponential smoothing to make forecasts for the time series of annual rainfall in London, we type:. The output of HoltWinters tells us that the estimated value of the alpha parameter is about 0. This is very close to zero, telling us that the forecasts are based on both recent and less recent observations although somewhat more weight is placed on recent observations. By default, HoltWinters just makes forecasts for the same time period covered by our original time series.

In this case, our original time series included rainfall for London from , so the forecasts are also for The plot shows the original time series in black, and the forecasts as a red line. The time series of forecasts is much smoother than the time series of the original data here.

As a measure of the accuracy of the forecasts, we can calculate the sum of squared errors for the in-sample forecast errors, that is, the forecast errors for the time period covered by our original time series. We prefer to think of it of an environment within which many classical and modern statistical techniques have been implemented. A few of these are built into the base R environment, but many are supplied as packages. More details on packages are given later see Packages.

Most classical statistics and much of the latest methodology is available for use with R, but users may need to be prepared to do a little work to find it. There is an important difference in philosophy between S and hence R and the other main statistical systems.

In S a statistical analysis is normally done as a series of steps, with intermediate results being stored in objects. Thus whereas SAS and SPSS will give copious output from a regression or discriminant analysis, R will give minimal output and store the results in a fit object for subsequent interrogation by further R functions.

The most convenient way to use R is at a graphics workstation running a windowing system. This guide is aimed at users who have this facility.

In particular we will occasionally refer to the use of R on an X window system although the vast bulk of what is said applies generally to any implementation of the R environment.

Most users will find it necessary to interact directly with the operating system on their computer from time to time. In this guide, we mainly discuss interaction with the operating system on UNIX machines. If you are running R under Windows or macOS you will need to make some small adjustments.

Setting up a workstation to take full advantage of the customizable features of R is a straightforward if somewhat tedious procedure, and will not be considered further here. Users in difficulty should seek local expert help. Next: Getting help with functions and features , Previous: R and the window system , Up: Introduction and preliminaries [ Contents ][ Index ]. When you use the R program it issues a prompt when it expects input commands. However, as we shall see, it is easy to change to a different R prompt if you wish.

At this point you will be asked whether you want to save the data from your R session. On some systems this will bring up a dialog box, and on others you will receive a text prompt to which you can respond yes , no or cancel a single letter abbreviation will do to save the data before quitting, quit without saving, or return to the R session.

Data which is saved will be available in future R sessions. To use R under Windows the procedure to follow is basically the same. Create a folder as the working directory, and set that in the Start In field in your R shortcut. Then launch R by double clicking on the icon. Readers wishing to get a feel for R at a computer before proceeding are strongly advised to work through the introductory session given in A sample session. Next: R commands, case sensitivity, etc.

R has an inbuilt help facility similar to the man facility of UNIX. To get more information on any specific named function, for example solve , the command is. Either form of quote mark may be used to escape the other, as in the string "It's important".

Our convention is to use double quote marks for preference. It can be a great way to get your bearings quickly and to understand the breadth of what R has to offer. The help. For example,. Next: Recall and correction of previous commands , Previous: Getting help with functions and features , Up: Introduction and preliminaries [ Contents ][ Index ].

Technically R is an expression language with a very simple syntax. It is case sensitive as are most UNIX based packages, so A and a are different symbols and would refer to different variables. The set of symbols which can be used in R names depends on the operating system and country within which R is being run technically on the locale in use.

Names are effectively unlimited in length. Elementary commands consist of either expressions or assignments. If an expression is given as a command, it is evaluated, printed unless specifically made invisible , and the value is lost. An assignment also evaluates an expression and passes the value to a variable but the result is not automatically printed.

This prompt may be changed by the user. We will generally omit the continuation prompt and indicate continuation by simple indenting. Command lines entered at the console are limited 4 to about bytes not characters. Next: Executing commands from or diverting output to a file , Previous: R commands, case sensitivity, etc. The vertical arrow keys on the keyboard can be used to scroll forward and backward through a command history. Once a command is located in this way, the cursor can be moved within the command using the horizontal arrow keys, and characters can be removed with the DEL key or added with the other keys.

More details are provided later: see The command-line editor. The recall and editing capabilities under UNIX are highly customizable. You can find out how to do this by reading the manual entry for the readline library. Next: Data permanency and removing objects , Previous: Recall and correction of previous commands , Up: Introduction and preliminaries [ Contents ][ Index ]. If commands 5 are stored in an external file, say commands.

R in the working directory work , they may be executed at any time in an R session with the command. For Windows Source is also available on the File menu. The function sink ,. The command. Previous: Executing commands from or diverting output to a file , Up: Introduction and preliminaries [ Contents ][ Index ]. The entities that R creates and manipulates are known as objects.

These may be variables, arrays of numbers, character strings, functions, or more general structures built from such components.

During an R session, objects are created and stored by name we discuss this process in the next section. The R command. The collection of objects currently stored is called the workspace. All objects created during an R session can be stored permanently in a file for use in future R sessions. At the end of each R session you are given the opportunity to save all the currently available objects. If you indicate that you want to do this, the objects are written to a file called. RData 6 in the current directory, and the command lines used in the session are saved to a file called.

When R is started at later time from the same directory it reloads the workspace from this file. At the same time the associated commands history is reloaded. It is recommended that you should use separate working directories for analyses conducted with R.

It is quite common for objects with names x and y to be created during an analysis. Names like this are often meaningful in the context of a single analysis, but it can be quite hard to decide what they might be when the several analyses have been conducted in the same directory. Next: Vector arithmetic , Previous: Simple manipulations; numbers and vectors , Up: Simple manipulations; numbers and vectors [ Contents ][ Index ].

R operates on named data structures. The simplest such structure is the numeric vector , which is a single entity consisting of an ordered collection of numbers. To set up a vector named x , say, consisting of five numbers, namely This is an assignment statement using the function c which in this context can take an arbitrary number of vector arguments and whose value is a vector got by concatenating its arguments end to end.

Assignment can also be made using the function assign. An equivalent way of making the same assignment as above is with:. Assignments can also be made in the other direction, using the obvious change in the assignment operator. So the same assignment could be made using.

If an expression is used as a complete command, the value is printed and lost 8. So now if we were to use the command.

Next: Generating regular sequences , Previous: Vectors and assignment , Up: Simple manipulations; numbers and vectors [ Contents ][ Index ]. Vectors can be used in arithmetic expressions, in which case the operations are performed element by element. Vectors occurring in the same expression need not all be of the same length. If they are not, the value of the expression is a vector with the same length as the longest vector which occurs in the expression.

Shorter vectors in the expression are recycled as often as need be perhaps fractionally until they match the length of the longest vector. In particular a constant is simply repeated. So with the above assignments the command. In addition all of the common arithmetic functions are available. If the argument to var is an n -by- p matrix the value is a p -by- p sample covariance matrix got by regarding the rows as independent p -variate sample vectors. Note that max and min select the largest and smallest values in their arguments, even if they are given several vectors.

The parallel maximum and minimum functions pmax and pmin return a vector of length equal to their longest argument that contains in each element the largest smallest element in that position in any of the input vectors.

Internally calculations are done as double precision real numbers, or double precision complex numbers if the input data are complex. R has a number of facilities for generating commonly used sequences of numbers.

For example is the vector c 1, 2, …, 29, The function seq is a more general facility for generating sequences. It has five arguments, only some of which may be specified in any one call. The first two arguments, if given, specify the beginning and end of the sequence, and if these are the only two arguments given the result is the same as the colon operator.

That is seq 2,10 is the same vector as Arguments to seq , and to many other R functions, can also be given in named form, in which case the order in which they appear is irrelevant.

A related function is rep which can be used for replicating an object in various complicated ways. The simplest form is. Another useful version is. As well as numerical vectors, R allows manipulation of logical quantities. The first two are often abbreviated as T and F , respectively. However there are situations where logical vectors and their coerced numeric counterparts are not equivalent, for example see the next subsection. In some cases the components of a vector may not be completely known.

In general any operation on an NA becomes an NA. The motivation for this rule is simply that if the specification of an operation is incomplete, the result cannot be known and hence is not available.

The function is. Examples are. In summary, is. To differentiate these, is. Next: Index vectors; selecting and modifying subsets of a data set , Previous: Missing values , Up: Simple manipulations; numbers and vectors [ Contents ][ Index ]. Character quantities and character vectors are used frequently in R, for example as plot labels. Where needed they are denoted by a sequence of characters delimited by the double quote character, e.

Character strings are entered using either matching double " or single ' quotes, but are printed using double quotes or sometimes without quotes. Quotes for a full list. Character vectors may be concatenated into a vector by the c function; examples of their use will emerge frequently. The paste function takes an arbitrary number of arguments and concatenates them one by one into character strings. Any numbers given among the arguments are coerced into character strings in the evident way, that is, in the same way they would be if they were printed.

Note particularly that recycling of short lists takes place here too; thus c "X", "Y" is repeated 5 times to match the sequence Subsets of the elements of a vector may be selected by appending to the name of the vector an index vector in square brackets. More generally any expression that evaluates to a vector may have subsets of its elements similarly selected by appending an index vector in square brackets immediately after the expression.

Note that if x has missing values, y will be shorter than x. The advantage is that alphanumeric names are often easier to remember than numeric indices. This option is particularly useful in connection with data frames, as we shall see later.

An indexed expression can also appear on the receiving end of an assignment, in which case the assignment operation is performed only on those elements of the vector.

Previous: Index vectors; selecting and modifying subsets of a data set , Up: Simple manipulations; numbers and vectors [ Contents ][ Index ]. Vectors are the most important type of object in R, but there are several others which we will meet more formally in later sections. Next: Changing the length of an object , Previous: Objects, their modes and attributes , Up: Objects, their modes and attributes [ Contents ][ Index ].

The entities R operates on are technically known as objects. Examples are vectors of numeric real or complex values, vectors of logical values and vectors of character strings. Vectors must have their values all of the same mode. Thus any given vector must be unambiguously either logical , numeric , complex , character or raw. Note that a vector can be empty and still have a mode. For example the empty character string vector is listed as character 0 and the empty numeric vector as numeric 0.

R also operates on objects called lists , which are of mode list. These are ordered sequences of objects which individually can be of any mode. The other recursive structures are those of mode function and expression. Functions are the objects that form part of the R system along with similar user written functions, which we discuss in some detail later. Expressions as objects form an advanced part of R which will not be discussed in this guide, except indirectly when we discuss formulae used with modeling in R.

By the mode of an object we mean the basic type of its fundamental constituents. Another property of every object is its length. The functions mode object and length object can be used to find out the mode and length of any defined structure Further properties of an object are usually provided by attributes object , see Getting and setting attributes.

For example, if z is a complex vector of length , then in an expression mode z is the character string "complex" and length z is R caters for changes of mode almost anywhere it could be considered sensible to do so, and a few where it might not be. For example with. A further coercion , or change of mode, reconstructs the numerical vector again:.

Now d and z are the same. The reader should consult the different help files to become familiar with them. Next: Getting and setting attributes , Previous: Intrinsic attributes: mode and length , Up: Objects, their modes and attributes [ Contents ][ Index ]. Similarly character is a empty character vector, and so on.

Once an object of any size has been created, new components may be added to it simply by giving it an index value outside its previous range. This applies to any structure at all, provided the mode of the additional component s agrees with the mode of the object in the first place. This automatic adjustment of lengths of an object is used often, for example in the scan function for input.

Conversely to truncate the size of an object requires only an assignment to do so. Hence if alpha is an object of length 10, then. The old indices are not retained, of course. We can then retain just the first three values by. Next: The class of an object , Previous: Changing the length of an object , Up: Objects, their modes and attributes [ Contents ][ Index ]. The function attributes object returns a list of all the non-intrinsic attributes currently defined for that object.

The function attr object , name can be used to select a specific attribute. These functions are rarely used, except in rather special circumstances when some new attribute is being created for some particular purpose, for example to associate a creation date or an operator with an R object.

The concept, however, is very important. Some care should be exercised when assigning or deleting attributes since they are an integral part of the object system used in R. When it is used on the left hand side of an assignment it can be used either to associate a new attribute with object or to change an existing one. For example. Previous: Getting and setting attributes , Up: Objects, their modes and attributes [ Contents ][ Index ].

All objects in R have a class , reported by the function class. For simple vectors this is just the mode, for example "numeric" , "logical" , "character" or "list" , but "matrix" , "array" , "factor" and "data. A special attribute known as the class of the object is used to allow for an object-oriented style 13 of programming in R. For example if an object has class "data. To remove temporarily the effects of class, use the function unclass.

For example if winter has the class "data. Only in rather special situations do you need to use this facility, but one is when you are learning to come to terms with the idea of class and generic functions. Generic functions and classes will be discussed further in Classes, generic functions and object orientation , but only briefly. A factor is a vector object used to specify a discrete classification grouping of the components of other vectors of the same length.

R provides both ordered and unordered factors. Suppose, for example, we have a sample of 30 tax accountants from all the states and territories of Australia 14 and their individual state of origin is specified by a character vector of state mnemonics as. A factor is similarly created using the factor function:. To find out the levels of a factor the function levels can be used. To continue the previous example, suppose we have the incomes of the same tax accountants in another vector in suitably large units of money.

To calculate the sample mean income for each state we can now use the special function tapply :. The function tapply is used to apply a function, here mean , to each group of components of the first argument, here incomes , defined by the levels of the second component, here statef 15 , as if they were separate vector structures. The result is a structure of the same length as the levels attribute of the factor containing the results.

The reader should consult the help document for more details. Suppose further we needed to calculate the standard errors of the state income means.

To do this we need to write an R function to calculate the standard error for any given vector. Since there is an builtin function var to calculate the sample variance, such a function is a very simple one liner, specified by the assignment:. Writing functions will be considered later in Writing your own functions. After this assignment, the standard errors are calculated by.

To do this you could use tapply once more with the length function to find the sample sizes, and the qt function to find the percentage points of the appropriate t -distributions. The function tapply can also be used to handle more complicated indexing of a vector by multiple categories.

For example, we might wish to split the tax accountants by both state and sex. However in this simple instance just one factor what happens can be thought of as follows. The values in the vector are collected into groups corresponding to the distinct entries in the factor.

The function is then applied to each of these groups individually. The value is a vector of function results, labelled by the levels attribute of the factor. The combination of a vector and a labelling factor is an example of what is sometimes called a ragged array , since the subclass sizes are possibly irregular. When the subclass sizes are all the same the indexing may be done implicitly and much more efficiently, as we see in the next section.

Previous: The function tapply and ragged arrays , Up: Ordered and unordered factors [ Contents ][ Index ]. The levels of factors are stored in alphabetical order, or in the order they were specified to factor if they were specified explicitly. Sometimes the levels will have a natural ordering that we want to record and want our statistical analysis to make use of.

The ordered function creates such ordered factors but is otherwise identical to factor. For most purposes the only difference between ordered and unordered factors is that the former are printed showing the ordering of the levels, but the contrasts generated for them in fitting linear models are different.

Next: Array indexing. An array can be considered as a multiply subscripted collection of data entries, for example numeric. R allows simple facilities for creating and handling arrays, and in particular the special case of matrices. A dimension vector is a vector of non-negative integers. If its length is k then the array is k -dimensional, e. The dimensions are indexed from one up to the values given in the dimension vector.

A vector can be used by R as an array only if it has a dimension vector as its dim attribute. Suppose, for example, z is a vector of elements.

The assignment. Other functions such as matrix and array are available for simpler and more natural looking assignments, as we shall see in The array function. Arrays can be one-dimensional: such arrays are usually treated in the same way as vectors including when printing , but the exceptions can cause confusion. Individual elements of an array may be referenced by giving the name of the array followed by the subscripts in square brackets, separated by commas.

More generally, subsections of an array may be specified by giving a sequence of index vectors in place of subscripts; however if any index position is given an empty index vector, then the full range of that subscript is taken.

For any array, say Z , the dimension vector may be referenced explicitly as dim Z on either side of an assignment. Also, if an array name is given with just one subscript or index vector , then the corresponding values of the data vector only are used; in this case the dimension vector is ignored.

This is not the case, however, if the single index is not a vector but itself an array, as we next discuss. Next: The array function , Previous: Array indexing. Subsections of an array , Up: Arrays and matrices [ Contents ][ Index ]. As well as an index vector in any subscript position, a matrix may be used with a single index matrix in order either to assign a vector of quantities to an irregular collection of elements in the array, or to extract an irregular collection as a vector.

A matrix example makes the process clear. In the case of a doubly indexed array, an index matrix may be given consisting of two columns and as many rows as desired. The entries in the index matrix are the row and column indices for the doubly indexed array. Suppose for example we have a 4 by 5 array X and we wish to do the following:. In this case we need a 3 by 2 subscript array, as in the following example. Negative indices are not allowed in index matrices. NA and zero values are allowed: rows in the index matrix containing a zero are ignored, and rows containing an NA produce an NA in the result.

As a less trivial example, suppose we wish to generate an unreduced design matrix for a block design defined by factors blocks b levels and varieties v levels. Further suppose there are n plots in the experiment.

We could proceed as follows:. However a simpler direct way of producing this matrix is to use table :. Index matrices must be numerical: any other form of matrix e. As well as giving a vector structure a dim attribute, arrays can be constructed from vectors by the array function, which has the form. If the size of h is exactly 24 the result is the same as. However if h is shorter than 24, its values are recycled from the beginning again to make it up to size 24 see Mixed vector and array arithmetic.

As an extreme but common example. At this point dim Z stands for the dimension vector c 3,4,2 , and Z[] stands for the data vector as it was in h , and Z[] with an empty subscript or Z with no subscript stands for the entire array as an array. Arrays may be used in arithmetic expressions and the result is an array formed by element-by-element operations on the data vector.

The dim attributes of operands generally need to be the same, and this becomes the dimension vector of the result.

So if A , B and C are all similar arrays, then. However the precise rule concerning mixed array and vector calculations has to be considered a little more carefully. The precise rule affecting element by element mixed calculations with vectors and arrays is somewhat quirky and hard to find in the references. From experience we have found the following to be a reliable guide.