Write a CSV to store in Google Cloud Storage

Question

Background: I\'m taking data in my Python/AppEngine project and creating a .tsv file so that I can create charts with d3.js. Right now I\'m writing the CSV with each page l

Answer 1

You're still attempting to create the writer on a response:

writer = csv.writer(self.response.out, delimiter='\t')

You need to write to the GCS file. Something like this:

    datalist = MyEntity.all()
    bucket_name = os.environ.get('BUCKET_NAME', app_identity.get_default_gcs_bucket_name())
    filename = os.path.join(bucket_name, 'myfile.tsv')
    write_retry_params = gcs.RetryParams(backoff_factor=1.1)
    gcs_file = gcs.open(filename, 'w', content_type='text/csv', retry_params=write_retry_params)
    writer = csv.writer(gcs_file, delimiter='\t')
    writer.writerow(['field1', 'field2'])
    for eachco in datalist:
        writer.writerow([eachco.variable1, eachco.variable2])
    gcs_file.close()

Notes:

• not actually tested
• I also adjusted the filename to use bucket_name
• if you do this in the get() request you may want to check if the file already exists and, if so, use it, otherwise you'd be still generating it at every request. Alternatively you could move this code on a task or in the .tsv upload handler.

Answer 2

The problem is that writer.writerow doesn't return anything. The return type will be None, and you are trying to write that into gcs_file.