Apache PIG - Get only date from TimeStamp

家住魔仙堡 提交于 2019-12-12 03:37:17

问题


I've the following code:

Data = load '/user/cloudera/' using PigStorage('\t') 
as
(   ID:chararray, 
    Time_Interval:chararray, 
    Code:chararray); 

transf = foreach Source_Data generate  (int) ID, 
                                   ToString( ToDate((long) Time_Interval), 'yyyy-MM-dd hh:ss:mm') as TimeStamp,
                        (int) Code; 

SPLIT transf INTO       Src25 IF (ToString(TimeStamp, 'yyyy-MM-dd')=='2016-07-25'),
                        Src26 IF (ToString(TimeStamp, 'yyyy-MM-dd')=='2016-07-26');


STORE Src25 INTO '/user/cloudera/2016-07-25' using PigStorage('\t');
STORE Src26 INTO '/user/cloudera/2016-07-26' using PigStorage('\t');

I want to split the files by date and the rules that I'm putting in Split statement it gives me error...

How can I transform TimeStamp (used on transf statement) in Date to make the comparasion?

Many thanks!


回答1:


After you get the datetime object from ToDate, use GetYear(),GetMonth(),GetDay() on the datetime object and use CONCAT to construct only the date.

transf = foreach Source_Data generate  
                   (int) ID, 
                   ToString( ToDate((long) Time_Interval), 'yyyy-MM-dd hh:ss:mm') as TimeStamp,
                   (int) Code;

transf_new = foreach transf generate
                     ID,
                     TimeStamp,
                     CONCAT(CONCAT(CONCAT(GetYear(TimeStamp),'-')),(CONCAT(GetMonth(TimeStamp),'-')),GetDay(TimeStamp)) AS Day,-- Note:Brackets might be slightly off but it should be like 'yyyy-MM-dd' format
                     Code;

-- Now use the new Day column to split the data
SPLIT transf_new INTO       Src25 IF (Day =='2016-07-25'),
                            Src26 IF (Day =='2016-07-26');


来源:https://stackoverflow.com/questions/38568361/apache-pig-get-only-date-from-timestamp

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!