hive-udf

My Spark job with HiveContext and Saxon working fine unless no UDFs defined in code. In case of UDF implementation - HiveContext initialization failed with error. I heard there are saxon\java8 incompability solved in saxon 9.5.1.5, which is not released yet as free version in central maven repository: Caused by: java.lang.RuntimeException: XPathFactory#newInstance() failed to create an XPathFactory for the default object model: http://java.sun.com/jaxp/xpath/dom with the XPathFactoryConfigurationException: javax.xml.xpath.XPathFactoryConfigurationException: java.util.ServiceConfigurationError:

I have a requirement to find median for multiple double datatype columns.Request suggestion to find the correct approach. Below is my sample dataset with one column. I am expecting the median value to be returned as 1 for my sample. scala> sqlContext.sql("select num from test").show(); +---+ |num| +---+ |0.0| |0.0| |1.0| |1.0| |1.0| |1.0| +---+ I tried the following options 1) Hive UDAF percentile, it worked only for BigInt. 2) Hive UDAT percentile_approx, but it does not work as expected (returns 0.25 vs 1). sqlContext.sql("select percentile_approx(num,0.5) from test").show(); +----+ | _c0| +

My Hive table has a date column with UTC date strings. I want to get all rows for a specific EST date. I am trying to do something like the below: Select * from TableName T where TO_DATE(ConvertToESTTimeZone(T.date)) = "2014-01-12" I want to know if there is a function for ConvertToESTTimeZone, or how I can achieve that? I tried the following but it doesnt work (my default timezone is CST): TO_DATE(from_utc_timestamp(T.Date) = "2014-01-12" TO_DATE( from_utc_timestamp(to_utc_timestamp (unix_timestamp (T.date), 'CST'),'EST')) Thanks in advance. Update: Strange behavior. When I do this: select