How to import a CSV file using Google Sheets API V4

Question

Background

I'm developing a Python 2.7 script that analyzes data from an SQL table and at the end, generates a CSV file.

Once the file is generated, I'm logging into my google sheet account and use the import option to import my CSV file into the google spreadsheet

The manual labor is kinda stupid and I wish to add this ability to my script.

Google Sheets API V4

So, I followed this guide, Python Quickstart and was able to complete all the steps.

Then I followed Google Sheets API reference and looked into Method: spreadsheets.create. If I understand correctly, it does not provides the options to import from a file.

It seems like there is no API for the import functionality.

Question

How to import a CSV file using Google Sheets API V4? Is their an example/reference that I'm missing?

Answer 1

You have two options for importing g CSV file. You can use the Drive API to create a spreadsheet from a CSV, or you can use the Sheets API to create an empty spreadsheet and then use spreadsheets.batchUpdate with a PasteDataRequest to add CSV data.

Answer 2

I like Burnash's gspread library, but the import_csv function in his answer is limited. It always starts the paste at A1 of the first worksheet (tab) and deletes all other tabs.

I needed to paste starting at a particular tab and cell, so I took Sam Berlin's suggestion to use a PasteDataRequest. Here's my function:

def pasteCsv(csvFile, sheet, cell):
    '''
    csvFile - path to csv file to upload
    sheet - a gspread.Spreadsheet object
    cell - string giving starting cell, optionally including sheet/tab name
      ex: 'A1', 'MySheet!C3', etc.
    '''
    if '!' in cell:
        (tabName, cell) = cell.split('!')
        wks = sheet.worksheet(tabName)
    else:
        wks = sheet.sheet1
    (firstRow, firstColumn) = gspread.utils.a1_to_rowcol(cell)

    with open(csvFile, 'r') as f:
        csvContents = f.read()
    body = {
        'requests': [{
            'pasteData': {
                "coordinate": {
                    "sheetId": wks.id,
                    "rowIndex": firstRow-1,
                    "columnIndex": firstColumn-1,
                },
                "data": csvContents,
                "type": 'PASTE_NORMAL',
                "delimiter": ',',
            }
        }]
    }
    return sheet.batch_update(body)

Note that I used a raw pasteData request rather than the higher-level update_cells method to take advantage of Google's automatic (correct) handling of input data that contains quoted strings, which may contain non-delimeter commas.

Answer 3

I've spent couple of hours trying to make any of the other answers work. Libraries do not explain the authentication well, and don't work with google-provided way of handling credentials. On the other hand, Sam's answer doesn't elaborate on the details of using the API, which might be confusing at times. So, here is a full recipe of uploading CSVs to gSheets. It uses both Sam's and CapoChino's answers plus some of my own research.

1. Authenticate/Setup. Generally, refer to the docs
   • Big blue button will get you credentials.json with no extra steps
   • quickstart.py can easily be adapted into authenticate.py
   • scopes should contain https://www.googleapis.com/auth/spreadsheets

Hopefully by now you have your credentials stored, so let's move to the actual code

2. Recipe that should work out of the box:

import pickle
from googleapiclient.discovery import build

SPREADSHEET_ID = '1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs74OgvE2upms' # Get this one from the link in browser
worksheet_name = 'Sheet2'
path_to_csv = 'New Folder/much_data.csv'
path_to_credentials = 'Credentials/token.pickle'


# convenience routines
def find_sheet_id_by_name(sheet_name):
    # ugly, but works
    sheets_with_properties = API \
        .spreadsheets() \
        .get(spreadsheetId=SPREADSHEET_ID, fields='sheets.properties') \
        .execute() \
        .get('sheets')

    for sheet in sheets_with_properties:
        if 'title' in sheet['properties'].keys():
            if sheet['properties']['title'] == sheet_name:
                return sheet['properties']['sheetId']


def push_csv_to_gsheet(csv_path, sheet_id):
    with open(csv_path, 'r') as csv_file:
        csvContents = csv_file.read()
    body = {
        'requests': [{
            'pasteData': {
                "coordinate": {
                    "sheetId": sheet_id,
                    "rowIndex": "0",  # adapt this if you need different positioning
                    "columnIndex": "0", # adapt this if you need different positioning
                },
                "data": csvContents,
                "type": 'PASTE_NORMAL',
                "delimiter": ',',
            }
        }]
    }
    request = API.spreadsheets().batchUpdate(spreadsheetId=SPREADSHEET_ID, body=body)
    response = request.execute()
    return response


# upload
with open(path_to_credentials, 'rb') as token:
    credentials = pickle.load(token)

API = build('sheets', 'v4', credentials=credentials)

push_csv_to_gsheet(
    csv_path=path_to_csv,
    sheet_id=find_sheet_id_by_name(worksheet_name)
)

Good thing about directly using batchUpdate is that it uploads thousands of rows in a second. On a low level gspread does the same and should be as performant. Also there is gspread-pandas.

p.s. the code is tested with python 3.5, but this thread seemed to be most appropriate to submit it to.

Answer 4

An alternative to Sam Berlin's answer, you can turn your CSV into a list of lists and set that to your POST payload.

Such a function looks something like this:

def preprocess(table):
    table.to_csv('pivoted.csv') # I use Pandas but use whatever you'd like
    _file = open('pivoted.csv')
    contents = _file.read()
    array = contents.split('\n')
    master_array = []
    for row in array:
        master_array.append(row.split(','))
    return master_array

That master array gets thrown into the following:

body = {
      'values': newValues
}

    result2 = service.spreadsheets().values().update(spreadsheetId=spreadsheetId, range=rangeName + str(len(values) + start + 1), valueInputOption="USER_ENTERED", body=body).execute()

It works just fine for me.

Answer 5

Another alternative to Sam Berlin's answer. If you're using Python, you can use the Drive API via gspread to import a CSV file. Here's an example:

import gspread

# Check how to get `credentials`:
# https://github.com/burnash/gspread

gc = gspread.authorize(credentials)

# Read CSV file contents
content = open('file_to_import.csv', 'r').read()

gc.import_csv('<SPREADSHEET_ID>', content)

Related question: Upload CSV to Google Sheets using gspread