Get a Google Cloud Storage file from its BlobKey

Question

I wrote a Google App Engine application that makes use of Blobstore to save programmatically-generated data. To do so, I used the Files API, which unfortunately has been deprecated in favor to Google Cloud Storage. So I'm rewriting my helper class to work with GCS.

I'd like to keep the interface as similar as possible as it was before, also because I persist BlobKeys in the Datastore to keep references to the files (and changing the model of a production application is always painful). When i save something to GCS, i retrieve a BlobKey with

BlobKey blobKey = blobstoreService.createGsBlobKey("/gs/" + fileName.getBucketName() + "/" + fileName.getObjectName());

as prescribed here, and I persist it in the Datastore.

So here's the question: the documentation tells me how to serve a GCS file with blobstoreService.serve(blobKey, resp); in a servlet response, BUT how can I retrieve the file content (as InputStream, byte array or whatever) to use it in my code for further processing? In my current implementation I do that with a FileReadChannel reading from an AppEngineFile (both deprecated).

Answer 1

Here is the code to open a Google Storage Object as Input Stream. Unfortunately, you have to use bucket name and object name and not the blob key

GcsFilename gcs_filename = new GcsFilename(bucket_name, object_name);
GcsService service = GcsServiceFactory.createGcsService();
ReadableByteChannel rbc = service.openReadChannel(gcs_filename, 0);
InputStream stream = Channels.newInputStream(rbc);

Answer 2

Given a blobKey, use the BlobstoreInputStream class to read the value from Blobstore, as described in the documentation:

BlobstoreInputStream in = new BlobstoreInputStream(blobKey); 

Answer 3

You can get the cloudstorage filename only in the upload handler (fileInfo.gs_object_name) and store it in your database. After that it is lost and it seems not to be preserved in BlobInfo or other metadata structures.

Google says: Unlike BlobInfo metadata FileInfo metadata is not persisted to datastore. (There is no blob key either, but you can create one later if needed by calling create_gs_key.) You must save the gs_object_name yourself in your upload handler or this data will be lost.

Sorry, this is a python link, but it should be easy to find something similar in java. https://developers.google.com/appengine/docs/python/blobstore/fileinfoclass

Answer 4

Here is the Blobstore approach (sorry, this is for Python, but I am sure you find it quite similar for Java):

blob_reader = blobstore.BlobReader(blob_key)
if blob_reader:
  file_content = blob_reader.read()