1. 技术文章
  2. Upload File to Google Cloud Storage via appengine

Upload File to Google Cloud Storage via appengine

由 心不动则不痛 提交于 2019-12-05 22:27:47

Finally I am able to uploading a file from client end to Google Cloud Storage via appengine.

I assume that before executing these steps you have following things ready

  • JSON file from your service account.
  • Created a default bucket.

Step 1: Make a Servlet like this

package XXXXXXXXX;

import java.io.IOException;
import java.io.InputStream;
import java.security.GeneralSecurityException;
import java.util.Arrays;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import com.google.api.client.http.InputStreamContent;
import com.google.api.services.storage.Storage;
import com.google.api.services.storage.model.ObjectAccessControl;
import com.google.api.services.storage.model.StorageObject;

import XXXXXXXXXXXXX.StorageFactory;

//@author Umesh Chauhan

/**
 * Save File to GCS
 *
 * @param fileName        File Name with format
 * @header Content-Type   "*/*"
 * @return file path
 * @throws Exception      Any Error during upload
 */
public class UploadFile extends HttpServlet
{

    private static final long serialVersionUID = 1L;
    private final String BUCKET = "YOUR BUCKET NAME";
    private int maxFileSize = 6 * 1024 * 1024;

    @Override
    protected void doOptions ( HttpServletRequest req, HttpServletResponse resp ) throws ServletException, IOException
    {
        // pre-flight request processing
        resp.setHeader ( "Access-Control-Allow-Origin", "*" );
        resp.setHeader ( "Access-Control-Allow-Methods", "*" );
        resp.setHeader ( "Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept" );
    }

    @Override
    public void doPost ( HttpServletRequest request, HttpServletResponse response )
            throws ServletException, java.io.IOException
    {

        try
        {
            String path = uploadFile ( request.getParameter ( "fileName" ), request.getContentType (),
                    request.getInputStream (), BUCKET, request.getInputStream ().available () );
            // Sending Response
            response.setStatus ( HttpServletResponse.SC_OK );
            response.getWriter ().write ( path );
            response.getWriter ().flush ();
            response.getWriter ().close ();

        }
        catch ( GeneralSecurityException e )
        {
            e.printStackTrace ();
        }
    }

    public String uploadFile ( String name, String contentType, InputStream input, String bucketName,
            int contentLength ) throws IOException, GeneralSecurityException
    {

        InputStreamContent contentStream = new InputStreamContent ( contentType, input );

        if ( contentLength < maxFileSize )
        {

            // It is done Automatically.
            /*
             * // Setting the length improves upload performance
             * contentStream.setLength ( contentLength );
             */

            StorageObject objectMetadata = new StorageObject ()
                    // Set the destination object name
                    .setName ( name )
                    // Set the access control list to publicly read-only
                    .setAcl ( Arrays.asList (
                            new ObjectAccessControl ().setEntity ( "allUsers" ).setRole ( "READER" ) ) );

            // Do the insert
            Storage client = StorageFactory.getService ();

            Storage.Objects.Insert insertRequest = client.objects ()
                    .insert ( bucketName, objectMetadata, contentStream );

            insertRequest.execute ();

            return "https://storage.cloud.google.com/" + BUCKET + "/" + name;
        }
        else
        {
            throw new GeneralSecurityException ( "File size canot be more then 6 MB !" );
        }
    }

    public void doGet ( HttpServletRequest request, HttpServletResponse response )
            throws ServletException, java.io.IOException
    {
        throw new ServletException ( "GET method used with " + getClass ().getName () + ": POST method required." );
    }

}

Step 2: Your Storage Factory

package XXXXXXXXXXXX;

import java.io.IOException;
import java.net.URL;
import java.security.GeneralSecurityException;
import java.util.Collection;

import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.api.client.googleapis.javanet.GoogleNetHttpTransport;
import com.google.api.client.http.HttpTransport;
import com.google.api.client.json.JsonFactory;
import com.google.api.client.json.jackson2.JacksonFactory;
import com.google.api.services.storage.Storage;
import com.google.api.services.storage.StorageScopes;

//@author Umesh Chauhan 

public class StorageFactory
{

    private static Storage instance = null;

    public static synchronized Storage getService () throws IOException, GeneralSecurityException
    {
        if ( instance == null )
        {
            instance = buildService ();
        }
        return instance;
    }

    private static Storage buildService () throws IOException, GeneralSecurityException
    {

        HttpTransport transport = GoogleNetHttpTransport.newTrustedTransport ();
        JsonFactory jsonFactory = new JacksonFactory ();

        GoogleCredential credential = GoogleCredential.fromStream (
                new URL ( "HERE GOES THE URL FOR YOUR SERVICE ACCOUNT JSON - I USED GOOGLE DRIVE DIRECT DOWNLOAD LINK TO MY JSON FILE" )
                        .openStream () );

        // Depending on the environment that provides the default credentials
        // (for
        // example: Compute Engine, App Engine), the credentials may require us
        // to
        // specify the scopes we need explicitly. Check for this case, and
        // inject
        // the Cloud Storage scope if required.
        if ( credential.createScopedRequired () )
        {
            Collection<String> scopes = StorageScopes.all ();
            credential = credential.createScoped ( scopes );
        }

        return new Storage.Builder ( transport, jsonFactory, credential ).setApplicationName ( "YOUR PROJECT NAME" ).build ();
    }
}

Step 3: Update you web.xml

    <servlet>
        <servlet-name>UploadFile</servlet-name>
        <servlet-class>PACKAGE.UploadFile</servlet-class>
    </servlet>

    <servlet-mapping>
        <servlet-name>UploadFile</servlet-name>
        <url-pattern>/uploadManager/UploadFile</url-pattern>  //Based on your original URL
    </servlet-mapping>

Here is the right way for uploading to storage according to the docs

import com.google.appengine.tools.cloudstorage.*;

public void doPost(HttpServletRequest req, HttpServletResponse resp) throws IOException {
  String buffer = "the data to be saved in GCS bla bla";
  GcsFileOptions instance = GcsFileOptions.getDefaultInstance();
  GcsFilename fileName = new GcsFilename("bucketName", "fileName");
  GcsOutputChannel outputChannel;
  gcsService.createOrReplace(fileName, instance, ByteBuffer.wrap(buffer.getBytes()));
}

you'll find the full code here

First you need to solve CORS problem in pre flight request processing, you need to do it on back end: on Google App Engine it done by adding doOptions method like:

@Override
protected void doOptions(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException 
{ 
    resp.setHeader("Access-Control-Allow-Origin", "*");
    resp.setHeader("Access-Control-Allow-Methods", "*");
    resp.setHeader("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept");
}

then you need to make sure that you send you request with header like Content-Type: multipart/form-data, otherwise your file will be encoded incorrectly. In angular2 request headers are set as third(optional) parameter in your post request like: 

let headers = new Headers();
headers.append('content-type', 'multipart/form-data');
http.post(url, body, {
                  headers:headers
                })

I'm sorry this does not directly address your question, but I want to point it out anyway. I recommend using one of your normal endpoints to generate a temporary upload URL that your Angular client can use to send the file directly to Cloud Storage, without going through your app. One way to do that is via the blobstore API, as described here. You can upload to Cloud storage via that API as described here (further down the same page).

This reduces the amount of upload code you need on your server, is not subject to the 32MB limit for GAE requests, and is in line with Google's recommendations (as seen in the documentation links above).

use com.google.cloud.storage

  • when running on appengine - no need for authentication

  • when running locally run gcloud auth application-default login command see authentication

    import com.google.cloud.storage.*;


public static void upload(String bucketName, String fileId, String content) throws IOException {
    Storage storage = StorageOptions.newBuilder().build().getService();

    BlobInfo fileInfo = BlobInfo.newBuilder(bucketName, fileId)
            .build();

    InputStream fileIS = IOUtils.toInputStream(content, "UTF-8");
    storage.create(fileInfo, fileIS);

}
标签
java
google-app-engine
google-cloud-storage
google-cloud-endpoints
易学教程内所有资源均来自网络或用户发布的内容，如有违反法律规定的内容欢迎反馈！
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!