Google Drive Upload returns 50x server errors intermittently

Question

I'm using the java google drive API to upload some files into various Google Drive folders. The user whose credentials I'm using has access to all the folders.

I am uploading around 100 PDFs and 100 CSV files from two different servers running CentOS. These are typically updates though I do occasionally insert the files. These files are between 35k and 700 bytes in size, so not large.

About a week ago, I started getting exceptions when uploading the CSV files (but not the PDF files). The exceptions look like this:

com.google.api.client.googleapis.json.GoogleJsonResponseException: 500 Internal Server Error
{
"code" : 500,
"errors" : [ {
"domain" : "global",
"message" : "Internal Error",
"reason" : "internalError"
} ],
"message" : "Internal Error"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:145)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)

This happens both when converting the csv file and when not converting (this answer implies that the conversion is the issue: Google Drive SDK - 500: Internal Server error: File uploads successfully most of the time ).

I have seen 503 errors as well:

com.google.api.client.googleapis.json.GoogleJsonResponseException: 503 Service Unavailable
{
  "code" : 503,
  "errors" : [ {
    "domain" : "global",
    "message" : "Backend Error",
    "reason" : "backendError"
  } ],
  "message" : "Backend Error"
}
        at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:145)
        at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
        at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
        at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:312)

The errors don't seem to go away--have tried at different times of the day and from different computers and with different credentials and to different folders (though always accessed using the same API project).

The Google status page says nothing about the Drive API being down: http://www.google.com/appsstatus#hl=en&v=status

The API console that I use to view number of API requests has no information.

Here's my environment information:

• java 1.6.0_25-b06
• google-api-services-drive: v2-rev126-1.18.0-rc
• using jackson: com.google.api.client.json.jackson.JacksonFactory
• com.google.api.client.http.javanet.NetHttpTransport()

I do exponential backoff as outlined here: https://developers.google.com/drive/web/handle-errors#implementing_exponential_backoff but only for 403 errors.

I've done some googling of the error message and looking at SO questions. This one indicates that the only response is to wait: 500 Internal Server Error v3 Upload API: GoogleJsonResponseException but it has been over a week.

I'm stumped. I'm posting here because this is apparently the best place to get support for Google Drive issues (https://developers.google.com/drive/support)

Update:

More details. I've tried:

• lowering the number of threads to 1 (I had up to 15 threads pushing files). No change, still get the error.
• only having the CSVs pushed (not the PDFs). No change, still get the error.
• setting 'create new revision' to false on update: https://developers.google.com/drive/v2/reference/files/update No change, still get the error.
• pushing the CSVs to different file names. This worked!
• deleting the CSV from the folder. This worked!
• adding in waits before calls to google, ranging from 200ms to 500ms

The PDFs which are uploading fine have two orders of magnitude fewer revisions (~20). The number of revisions on some of the csv files is up to 2000+. I'm not sure if that is an issue or not.

Answer 1

One work around is to delete the CSV files via the web interface. Make sure you delete them entirely, as just moving them to the trash is insufficient.