hiveql

in Hive I'd like to dynamically extract information from a table, save it in a variable and further use it. Consider the following example, where I retrieve the maximum of column var and want to use it as a condition in the subsequent query. set maximo=select max(var) from table; select * from table where var=${hiveconf:maximo} It does not work, although set maximo=select max(var) from table; ${hiveconf:maximo} shows me the intended result. Doing: select '${hiveconf:maximo}' gives "select max(var) from table" though. Best Hive substitutes variables as is and does not execute them. Use shell

we would like to put the results of a Hive query to a CSV file. I thought the command should look like this: insert overwrite directory '/home/output.csv' select books from table; When I run it, it says it completeld successfully but I can never find the file. How do I find this file or should I be extracting the data in a different way? Thanks! Although it is possible to use INSERT OVERWRITE to get data out of Hive, it might not be the best method for your particular case. First let me explain what INSERT OVERWRITE does, then I'll describe the method I use to get tsv files from Hive tables.

I am having a Hive query like the one below: select a.x as column from table1 a where a.y in (<long comma-separated list of parameters>) union all select b.x as column from table2 b where b.y in (<long comma-separated list of parameters>) I have set hive.exec.parallel as true which is helping me achieve parallelism between the two queries between union all. But, my IN clause has many comma separated values and each value is taken once in 1 job and then the next value. This is actually getting executed sequentially. Is there any hive parameter which if enabled can help me fetch data parallelly

I am using Spark SQL (I mention that it is in Spark in case that affects the SQL syntax - I'm not familiar enough to be sure yet) and I have a table that I am trying to re-structure, but I'm getting stuck trying to transpose multiple columns at the same time. Basically I have data that looks like: userId someString varA varB 1 "example1" [0,2,5] [1,2,9] 2 "example2" [1,20,5] [9,null,6] and I'd like to explode both varA and varB simultaneously (the length will always be consistent) - so that the final output looks like this: userId someString varA varB 1 "example1" 0 1 1 "example1" 2 2 1

Can anyone tell me the difference between Hive's external table and internal tables. I know the difference comes when dropping the table. I don't understand what you mean by the data and metadata is deleted in internal and only metadata is deleted in external tables. Can anyone explain me in terms of nodes please. prestomation Hive has a relational database on the master node it uses to keep track of state. For instance, when you CREATE TABLE FOO(foo string) LOCATION 'hdfs://tmp/'; , this table schema is stored in the database. If you have a partitioned table, the partitions are stored in the

I'm looking for the SQL equivalent of SET varname = value in Hive QL I know I can do something like this: SET CURRENT_DATE = '2012-09-16'; SELECT * FROM foo WHERE day >= @CURRENT_DATE But then I get this error: character '@' not supported here libjack You need to use the special hiveconf for variable substitution. e.g. hive> set CURRENT_DATE='2012-09-16'; hive> select * from foo where day >= '${hiveconf:CURRENT_DATE}' similarly, you could pass on command line: % hive -hiveconf CURRENT_DATE='2012-09-16' -f test.hql Note that there are env and system variables as well, so you can reference ${env