hiveql

I have two tables with a single key column. Keys in table a are subset of all keys in table b. I need to select keys from table b that are NOT in table a. Here is a citation from Hive manual: "LEFT SEMI JOIN implements the uncorrelated IN/EXISTS subquery semantics in an efficient way. As of Hive 0.13 the IN/NOT IN/EXISTS/NOT EXISTS operators are supported using subqueries so most of these JOINs don't have to be performed manually anymore. The restrictions of using LEFT SEMI JOIN is that the right-hand-side table should only be referenced in the join condition (ON-clause), but not in WHERE- or

I want to create an external Hive table from a text file containing epoch in HDFS. Let's say the file is located at the /user/me/test.txt . Here's the file content: 1354183921 1354183922 I have Hive 0.8.1 installed and should be able to use type Timestamp, so I created the table: hive> CREATE EXTERNAL TABLE test1 (epoch Timestamp) LOCATION '/user/me'; Then I queried the table: SELECT * FROM test1; and got the following exception: Failed with exception java.io.IOException:java.lang.IllegalArgumentException: Timestamp format must be yyyy-mm-dd hh:mm:ss[.fffffffff] Have I missed anything when

I have a table having the following columns. C1,C2,Process TimeStamp,InsertDateTimeStamp p1,v1,2014-01-30 12:15:23,2013-10-01 05:34:23 p1,v2,2014-01-31 05:11:34,2013-12-01 06:12:31 p1,v3,2014-01-31 07:16:05,2012-09-01 07:45:20 p2,v4,2014-02-01 09:22:52,2013-12-01 06:12:31 p2,v5,2014-02-01 09:22:52,2012-09-01 07:45:20 Now, I want to fetch unique row for each primary key based on latest Process TimeStamp . If Process TimeStamp is same then row having latest InsertDateTimeStamp should be chosen. So, my result should be. p1,v3,2014-01-31 07:16:05,2012-09-01 07:45:20 p2,v4,2014-02-01 09:22:52,2013

Im new to Bigdata and currently learning Hive. I understood the concept of InputFormat & OutputFormat in Hive as part of SerDe. I also understood that 'Stored as' is used to store a file in a particular format just like InputFormat. But I don't understand what is the significant difference between using the 'InputFormat, OutputFormat' & 'Stored as'. Any help is appreciated. Hive has a lot of options of how to store the data. You can either use external storage where Hive would just wrap some data from other place or you can create standalone table from start in hive warehouse . Input and

I have the following string representation of a timestamp in my Hive table: 20130502081559999 I need to convert it to a string like so: 2013-05-02 08:15:59 I have tried following ({code} >>> {result}): from_unixtime(unix_timestamp('20130502081559999', 'yyyyMMddHHmmss')) >>> 2013-05-03 00:54:59 from_unixtime(unix_timestamp('20130502081559999', 'yyyyMMddHHmmssMS')) >>> 2013-09-02 08:15:59 from_unixtime(unix_timestamp('20130502081559999', 'yyyyMMddHHmmssMS')) >>> 2013-05-02 08:10:39 Converting to a timestamp and then unixtime seems weird, what is the proper way to do this? EDIT I figured it out.