Loop to list files in parts

问题

I'm trying to copy around 10,000 files and paste it into another directory with the following code:

f <- c (MYFILE $ COL1) # PART 1
m <- paste0 ("(", paste (f, collapse = "|"), ") .xml") #PART 2
files <- list.files (pattern = m) #PART 3

file.copy (paste0 (getwd (), "/", files), paste0 (getwd (), "/ FILES_XML /", files), overwrite = TRUE) # PART 4

When I try to run the files <- list.files (pattern = m) part, the following message appears:

assertion 'tree-> num_tags == num_tags' failed in executing regexp: file 'tre-compile.c', line 634

To prevent the message from appearing, I thought about running the code in 500 parts in 500. How would I do it in parts using for?

回答1:

Maybe something like the following will do it.
Note the use of help("file.path") as a replacement for paste.

n <- length(f)
sp <- rep(1:ceiling(n / 5), each = 5, length.out = n)
lapply(split(f, sp), function(x){
  m <- paste0 ("(", paste (x, collapse = "|"), ")\\.xml")
  files <- list.files(pattern = m)
  infiles <- file.path(getwd(), files)
  outfiles <- file.path(getwd(), "FILES_XML", files)
  file.copy(infiles, outfiles, overwrite = TRUE)
})

回答2:

Well even though I don't understand your reasoning, how this would prevvent the error message from appearing (?) here is how to run the file.copy part in a loop:

lower <- 1
for (upper in seq(500, 10000, by=500)) {
  file.copy(paste0(getwd (), "/", files[lower:upper]), paste0 (getwd (), "/ FILES_XML /", files[lower:upper]), overwrite = TRUE)
  lower <- upper + 1
}

回答3:

This is a regex error that probably comes from the line files <- list.files(pattern = m), which wouldn't be fixed by chunking the copying process. I bet you can fix it by thinking more carefully when constructing the search string m.

来源：https://stackoverflow.com/questions/60119882/loop-to-list-files-in-parts

标签

for-loop