How can I read the files in a directory in sorted order using R?

Question

The code given below worked well and read the files in my directory and extracted the values:

X <- c(75:85) ; Y <- c(208:215) 
extract <- vector()
f         


        

Answer 1

gtools package has the function "mixedsort" that can help you.

The example myFiles was taken from Peter Verbeet answer.

myFiles <- paste("Climate_Rad_", c(1:15, 95:110), ".img", sep = "") 

install.packages ('gtools')
require ('gtools')

mixedsort (myFiles)

[1] "Climate_Rad_1.img"   "Climate_Rad_2.img"  
[3] "Climate_Rad_3.img"   "Climate_Rad_4.img"  
[5] "Climate_Rad_5.img"   "Climate_Rad_6.img"  
[7] "Climate_Rad_7.img"   "Climate_Rad_8.img"  
[9] "Climate_Rad_9.img"   "Climate_Rad_10.img" 
[11] "Climate_Rad_11.img"  "Climate_Rad_12.img" 
[13] "Climate_Rad_13.img"  "Climate_Rad_14.img" 
[15] "Climate_Rad_15.img"  "Climate_Rad_95.img" 
[17] "Climate_Rad_96.img"  "Climate_Rad_97.img" 
[19] "Climate_Rad_98.img"  "Climate_Rad_99.img" 
[21] "Climate_Rad_100.img" "Climate_Rad_101.img"
[23] "Climate_Rad_102.img" "Climate_Rad_103.img"
[25] "Climate_Rad_104.img" "Climate_Rad_105.img"
[27] "Climate_Rad_106.img" "Climate_Rad_107.img"
[29] "Climate_Rad_108.img" "Climate_Rad_109.img"
[31] "Climate_Rad_110.img"

Answer 2

Why not retrieve all files (with a particular pattern) with list.files() and then sort that. Then you retrieve the files from the sorted vector, which gives them to you in correctly sorted order. This has the advantage that it also works when numbers are missing from the 1:365 sequence

Something like:

myFiles <- list.files(pattern = "^Climate_Rad_") #all files starting with Climate_
myFiles <- sort(myFiles)
# then read them in, for instance through
for (fileNames in myFiles) {READ.IN.AND.DO.YOUR.MAGIC.ON.THEM}

Answer 3

As I mentioned in the comment above, you can use paste to simplify your code a bit. Try:

# set the working directory
setwd("C:New folder (10)")

# construct a vector of file names and loop through
for (f in paste("Climate_Rad_", 1:365, ".img", sep="")) {
    conne <- file(f, "rb")
    # do rest, assuming it's correct
}

Answer 4

Ah, OK, I see, the issue is in the sorting. The sorting order is alphabetically correct. Climate_Rad_1 is alphabetically followed by Climate_Rad_10, rather than by Climate_Rad_2 The order is not "random" at all, it is alphabetically correct.

However, you want Climate_Rad_2 to be processed before Climate_Rad_10, not after. There are several ways to deal with that. First, you should note that Climate_Rad_002 does alphabeticaly come before Climate_Rad_010, so if you add leading zeros at the time that you generate the files, this will make it easy to process the files in numeric order later.

Alternatively, let's assume you werent able to add the zeros at the time of creating the files. Then there are at least two ways to still access the files in order. Either by adding the zeros to the filenames afterwards, or just sort on the numeric part of the filename.

Let me show you the latter.

myFiles <- paste("Climate_Rad_", c(1:15, 95:110), ".img", sep = "") # create some test names, you get the actual myFiles through a call to list.files()

myFiles.sorted <- sort(myFiles) # this gives the alphabetic sorting, not what you want

> myFiles.sorted
 [1] "Climate_Rad_1.img"   "Climate_Rad_10.img"  "Climate_Rad_100.img"
 [4] "Climate_Rad_101.img" "Climate_Rad_102.img" "Climate_Rad_103.img"
 [7] "Climate_Rad_104.img" "Climate_Rad_105.img" "Climate_Rad_106.img"
[10] "Climate_Rad_107.img" "Climate_Rad_108.img" "Climate_Rad_109.img"
[13] "Climate_Rad_11.img"  "Climate_Rad_110.img" "Climate_Rad_12.img" 
[16] "Climate_Rad_13.img"  "Climate_Rad_14.img"  "Climate_Rad_15.img" 
[19] "Climate_Rad_2.img"   "Climate_Rad_3.img"   "Climate_Rad_4.img"  
[22] "Climate_Rad_5.img"   "Climate_Rad_6.img"   "Climate_Rad_7.img"  
[25] "Climate_Rad_8.img"   "Climate_Rad_9.img"   "Climate_Rad_95.img" 
[28] "Climate_Rad_96.img"  "Climate_Rad_97.img"  "Climate_Rad_98.img" 
[31] "Climate_Rad_99.img" 

# split between the part that comes before the numerics and the "1.img" etc.--adjust appropriately
split <- strsplit(myFiles.sorted, "Climate_Rad_") 
# strip the "1.img" etc such that only the numeric part is left
# turn the characters in numeric
split <- as.numeric(sapply(split, function(x) x <- sub(".img", "", x[2])))
# not you can sort, by using order, that gives the original filenames, ordered on the numeric part of the filename
myFiles.correct.order <- myFiles.sorted[order(split)]

 [1] "Climate_Rad_1.img"   "Climate_Rad_2.img"   "Climate_Rad_3.img"  
 [4] "Climate_Rad_4.img"   "Climate_Rad_5.img"   "Climate_Rad_6.img"  
 [7] "Climate_Rad_7.img"   "Climate_Rad_8.img"   "Climate_Rad_9.img"  
[10] "Climate_Rad_10.img"  "Climate_Rad_11.img"  "Climate_Rad_12.img" 
[13] "Climate_Rad_13.img"  "Climate_Rad_14.img"  "Climate_Rad_15.img"   
[16] "Climate_Rad_95.img"  "Climate_Rad_96.img"  "Climate_Rad_97.img" 
[19] "Climate_Rad_98.img"  "Climate_Rad_99.img"  "Climate_Rad_100.img"
[22] "Climate_Rad_101.img" "Climate_Rad_102.img" "Climate_Rad_103.img"
[25] "Climate_Rad_104.img" "Climate_Rad_105.img" "Climate_Rad_106.img"
[28] "Climate_Rad_107.img" "Climate_Rad_108.img" "Climate_Rad_109.img"
[31] "Climate_Rad_110.img"

That will give you the files in the order you were looking for. Now pull the files according to that, e.g. by

for (fileNames in myFiles.correct.order) {READ.IN.AND.DO.YOUR.THING}

That should do it. Make sure you adjust the "Climate_Rad_" and ".img" as appropriate to your filenames (you may also have to add a path before the "Climate_Rad_", to make it something like "C:/filefolder/Climate_Rad_", if that is needed).