hiveql

For example I have few Hive HQL statements which I want to pass into Spark SQL: set parquet.compression=SNAPPY; create table MY_TABLE stored as parquet as select * from ANOTHER_TABLE; select * from MY_TABLE limit 5; Following doesn't work: hiveContext.sql("set parquet.compression=SNAPPY; create table MY_TABLE stored as parquet as select * from ANOTHER_TABLE; select * from MY_TABLE limit 5;") How to pass the statements into Spark SQL? Thank you to @SamsonScharfrichter for the answer. This will work: hiveContext.sql("set spark.sql.parquet.compression.codec=SNAPPY") hiveContext.sql("create table

I am trying different ways to query a record within a array of records and display complete Row as output. I dont know which nested Object has String "pg". But i want to query on particular object. Whether the object has "pg" or not. If "pg" exist then i want to display that complete row. How to write "spark sql query" on nested objects without specfying the object index.So i dont want to use the index of children.name My Avro Record: { "name": "Parent", "type":"record", "fields":[ {"name": "firstname", "type": "string"}, { "name":"children", "type":{ "type": "array", "items":{ "name":"child",

I have written a query to find 10 most busy airports in the USA from March to April. It produces the desired output however I want to try to further optimize it. Are there any HiveQL specific optimizations that can be applied to the query? Is GROUPING SETS applicable here? I'm new to Hive and for now this is the shortest query that I've come up with. SELECT airports.airport, COUNT(Flights.FlightsNum) AS Total_Flights FROM ( SELECT Origin AS Airport, FlightsNum FROM flights_stats WHERE (Cancelled = 0 AND Month IN (3,4)) UNION ALL SELECT Dest AS Airport, FlightsNum FROM flights_stats WHERE

I'm running a hive query against a hadoop cluster of 3 nodes. And I am getting an error which says "Too many fetch failures". My hive query is: insert overwrite table tablename1 partition(namep) select id,name,substring(name,5,2) as namep from tablename2; that's the query im trying to run. All i want to do is transfer data from tablename2 to tablename1. Any help is appreciated. javadba This can be caused by various hadoop configuration issues. Here a couple to look for in particular: DNS issue : examine your /etc/hosts Not enough http threads on the mapper side for the reducer Some suggested

On Hive 2.2.0, I am filling an orc table from another source table of size 1.34 GB, using the query INSERT INTO TABLE TableOrc SELECT * FROM Table; ---- (1) The query creates TableORC table with 6 orc files, which are much smaller than the block size of 256MB. -- FolderList1 -rwxr-xr-x user1 supergroup 65.01 MB 1/1/2016, 10:14:21 AM 1 256 MB 000000_0 -rwxr-xr-x user1 supergroup 67.48 MB 1/1/2016, 10:14:55 AM 1 256 MB 000001_0 -rwxr-xr-x user1 supergroup 66.3 MB 1/1/2016, 10:15:18 AM 1 256 MB 000002_0 -rwxr-xr-x user1 supergroup 63.83 MB 1/1/2016, 10:15:41 AM 1 256 MB 000003_0 -rwxr-xr-x user1

I am using Hive version 0.13.1. While trying to insert data into an existing table getting an error while using the below query: CREATE TABLE table1 (order_num int, payment_type varchar(20), category varchar(20)); INSERT INTO TABLE table1 VALUES (151, 'cash', 'lunch'); ERROR : ParseException line 1:25 cannot recognize input near 'VALUES' '(' '151' in select clause While searching, got everyone suggesting above query, but unfortunately it's not working for me. Is it due to different Hive version? I am getting this ambiguity due to link here Needs help to insert data to an existing table in Hive