SPARK SQL - update MySql table using DataFrames and JDBC

Question

I\'m trying to insert and update some data on MySql using Spark SQL DataFrames and JDBC connection.

I\'ve succeeded to insert new data using the SaveMode.Append. Is

Answer 1

In PYSPARK I was not able to do that so I decided to use odbc.

url = "jdbc:sqlserver://xxx:1433;databaseName=xxx;user=xxx;password=xxx"
df.write.jdbc(url=url, table="__TableInsert", mode='overwrite')
cnxn  = pyodbc.connect('Driver={ODBC Driver 17 for SQL Server};Server=xxx;Database=xxx;Uid=xxx;Pwd=xxx;', autocommit=False) 
try:
    crsr = cnxn.cursor()
    # DO UPSERTS OR WHATEVER YOU WANT
    crsr.execute("DELETE FROM Table")
    crsr.execute("INSERT INTO Table (Field) SELECT Field FROM __TableInsert")
    cnxn.commit()
except:
    cnxn.rollback()
cnxn.close()