R reactiveFileReader reading from aws s3 bucket

问题

I can read a csv from my S3 bucket using the below code

aws.s3::s3read_using(read.csv,
                     stringsAsFactors=FALSE,
                     check.names=FALSE,
                     object=paste0(Sys.getenv("BUCKET_PREFIX"), "/a.csv"),
                     bucket = Sys.getenv("AWS_BUCKET_NAME"),
                     opts=bucket_opts
    )

I want to change this to using the function reactiveFileReader. I tried the below with no success, any idea what I am doing wrong?

reactiveFileReader(
      intervalMillis = 10000,
      session= session,
      filePath = paste0(Sys.getenv("BUCKET_PREFIX"), "/a.csv"),
      readFunc = aws.s3::s3read_using,
      FUN = read.csv,
      stringsAsFactors=FALSE,
      check.names=FALSE,
      object=paste0(Sys.getenv("BUCKET_PREFIX"), "/a.csv"),
      bucket = Sys.getenv("AWS_BUCKET_NAME"),
      opts=bucket_opts
      )

回答1:

I had the same problem and solved it with a bit of a workaround:

in global.R you read your data at start of shiny app.

data_obj <- aws.s3::s3read_using(read.csv,
                     stringsAsFactors=FALSE,
                     check.names=FALSE,
                     object=paste0(Sys.getenv("BUCKET_PREFIX"), "/a.csv"),
                     bucket = Sys.getenv("AWS_BUCKET_NAME"),
                     opts=bucket_opts
    )

#bucket information
content_bucket <- aws.s3::get_bucket(bucket = Sys.getenv("AWS_BUCKET_NAME"))

in server.R you set a timer to check if data on S3 changed.

  shiny::observe({

shiny::invalidateLater(3600e3) #timer for every hour to check the following

#temporary bucket information
bucket_content_check <- aws.s3::get_bucket(bucket = Sys.getenv("AWS_BUCKET_NAME"))

#reload data if timestamp differs
    if (bucket_content$Contents$LastModified != bucket_content_check$Contents$LastModified) {
      
data_obj <<- aws.s3::s3read_using(read.csv,
                     stringsAsFactors=FALSE,
                     check.names=FALSE,
                     object=paste0(Sys.getenv("BUCKET_PREFIX"), "/a.csv"),
                     bucket = Sys.getenv("AWS_BUCKET_NAME"),
                     opts=bucket_opts
    )

#overwrite global bucket_content for next check
content_bucket <<- aws.s3::get_bucket(bucket = Sys.getenv("AWS_BUCKET_NAME"))

message('S3 data reloaded')
      }

    message('S3 bucket checked')
})

来源：https://stackoverflow.com/questions/63673007/r-reactivefilereader-reading-from-aws-s3-bucket

标签

amazon-s3

Shiny