Appending pandas Data Frame to Google spreadsheet

Question

Case: My script returns a data frame that needs has to be appended to an existing google spreadsheet as new rows of data.As of now, I\'m appending a data frame as multiple s

Answer 1

Here is the code to write, append(without loading the existing sheet into memory), and read to google sheets.

import gspread_dataframe as gd
import gspread as gs
gc = gs.service_account(filename="your/cred/file.json")

def export_to_sheets(worksheet_name,df,mode='r'):
    ws = gc.open("SHEET_NAME").worksheet("worksheet_name")
    if(mode=='w'):
        ws.clear()
        gd.set_with_dataframe(worksheet=ws,dataframe=df,include_index=False,include_column_header=True,resize=True)
        return True
    elif(mode=='a'):
        ws.add_rows(df.shape[0])
        gd.set_with_dataframe(worksheet=ws,dataframe=df,include_index=False,include_column_header=False,row=ws.row_count+1,resize=False)
        return True
    else:
        return gd.get_as_dataframe(worksheet=ws)
    
df = pd.DataFrame.from_records([{'a': i, 'b': i * 2} for i in range(100)])
export_to_sheets("SHEET_NAME",df,'a')

---

1. Write Mode: First clear existing worksheet => ws.clear() .Second using set_with_dataframe() uploading the dataframe, here note that resize=True, which strictily set the row and col in worksheet to df.shape. This will help later in append method.
2. Append Mode: First, add rows according to the dataframe. Second setting the parameter resize=False as we are adding rows and row=ws.row_count+1 anchoring its row value for append.
3. Read Mode(Default): returns a dataframe

Answer 2

ws = gc.open("sheet title").worksheet("ws")

gd.set_with_dataframe(ws, dataframe)

#simply transform your dataframe to google sheet

Answer 3

if Google spreadsheet takes .csv format then you can convert a pandas dataframe to csv using df.to_csv() and save it in that format

Answer 4

I came up with the following solution. It does not overwrite current data but just appends entire pandas DataFrame df to the end of Sheet with name sheet in the Spreadsheet with the name spread_sheet.

import gspread
from google.auth.transport.requests import AuthorizedSession
from oauth2client.service_account import ServiceAccountCredentials

def append_df_to_gs(df, spread_sheet:str, sheet_name:str):
    scopes = [
        'https://spreadsheets.google.com/feeds',
        'https://www.googleapis.com/auth/drive',
    ]
    credentials = ServiceAccountCredentials.from_json_keyfile_name(
        path_to_credentials,
        scopes=scopes
    )
    gsc = gspread.authorize(credentials)
    sheet = gsc.open(spread_sheet)
    params = {'valueInputOption': 'USER_ENTERED'}
    body = {'values': df.values.tolist()}
    sheet.values_append(f'{sheet_name:str}!A1:G1', params, body)

For params valueInputOption please consult this. I used USER_ENTERED here as I needed some formulas to be valid once I append the data to Google Sheets.

Answer 5

I can recommend gspread-dataframe:

import gspread_dataframe as gd

# Connecting with `gspread` here

ws = gc.open("SheetName").worksheet("xyz")
existing = gd.get_as_dataframe(ws)
updated = existing.append(your_new_data)
gd.set_with_dataframe(ws, updated)