Which is the best method to apply a script repetitively to n .csv files in R?

Question

My situation:

1. I have a number of csv files all with the same suffix pre .csv, but the first two characters of the file name are different (ie AA01.csv, AB01.csv, AC01.csv etc)
2. I have an R script which I would like to run on each file. This file essentially extracts the data from the .csv and assigns them to vectors / converts them into timeseries objects. (For example, AA01 xts timeseries object, AB01 xts object)

What I would like to achieve:

1. Embed the script within a larger loop (or as appropriate) to sequentially run over each file and apply the script
2. Remove the intermediate objects created (see code snippet below)
3. Leave me with the final xts objects created from each raw data file (ie AA01 to AC01 etc as Values / Vectors etc)

What would be the right way to embed this script in R? Sorry, but I am a programming noob!

My script code below...heading of each column in each CSV is DATE, TIME, VALUE

    # Pull in Data from the FileSystem and attach it
AA01raw<-read.csv("AA01.csv")
attach(AA01raw)
#format the data for timeseries work
cdt<-as.character(Date)
ctm<-as.character(Time)
tfrm<-timeDate(paste(cdt,ctm),format ="%Y/%m/%d %H:%M:%S")
val<-as.matrix(Value)
aa01tsobj<-timeSeries(val,tfrm)
#convert the timeSeries object to an xts Object
aa01xtsobj<-as.xts(tsobj)
#remove all the intermediate objects to leave the final xts object
rm(cdt)
rm(ctm)
rm(aa01tsobj)
rm(tfrm)
gc()

and then repeat on each .csv file til all xts objects are extracted.

ie, what we would end up within R, ready for further applications are:

aa01xtsobj, ab01xtsobj, ac01xtsobj....etc

any help on how to do this would be very much appreciated.

Answer 1

I find a for loop and lists is well enough for stuff like this. Once you have a working set of code it's easy enough to move from a loop into a function which can be sapplyied or similar, but that kind of vectorization is idiosyncratic anyway and probably not useful outside of private one-liners.

You probably want to avoid assigning to multiple objects with different names in the workspace (this a FAQ which usually comes up as "how do I assign() . . .").

Please beware my untested code.

A vector of file names, and a list with a named element for each file.

files <- c("AA01.csv", "AA02.csv")
lst <- vector("list", length(files))
names(lst) <- files

Loop over each file.

library(timeSeries)

for (i in 1:length(files)) {
    ## read strings as character
    tmp <- read.csv(files[i], stringsAsFactors = FALSE)
    ## convert to 'timeDate'
    tmp$tfrm <- timeDate(paste(tmp$cdt, tmp$ctm),format ="%Y/%m/%d %H:%M:%S"))
    ## create timeSeries object
    obj <- timeSeries(as.matrix(tmp$Value), tmp$tfrm)
    ## store object in the list, by name
    lst[[files[i]]] <- as.xts(obj)
}

## clean up
rm(tmp, files, obj)

Now all the read objects are in lst, but you'll want to test that the file is available, that it was read correctly, and you may want to modify the names to be more sensible than just the file name.

Print out the first object by name index from the list:

lst[[files[1]]]

Answer 2

Be sure to use Rs dir command to produce the list of filenames instead of manually entering them in.

filenames = dir(pattern="*01.csv")
for( i in 1:length(filenames) )
{
  ...