“INSERT INTO …” with SparkSQL HiveContext

Question

I\'m trying to run an insert statement with my HiveContext, like this:

hiveContext.sql(\'insert into my_table (id, score) values (1, 10)\')

Th

Answer 1

You tried to perform something that the data file format cannot, hence the Unsupported language features in query exception.

Many data file format are write-once and no support ACID operation.

Apache ORC supports ACID operation if you need it.

Instead, you can use partition to split your data into folders (/data/year=2017/month=10....), here you can append/insert data into your data lake.