1 ,库存流水,转换格式 :
spark-submit --master yarn --deploy-mode cluster --num-executors 5 --executor-cores 3 --executor-memory 6144m --class lifecycle01_tool.ParseCsvToParquet s3://lifecyclebigdata/dataWareHouse/BALABALA/00jar/02_work/veryOK-1.0-SNAPSHOT.jar lifecyclebigdata/dataWareHouse/BALABALA/01history/2019Q3_3/库存流水.gz lifecyclebigdata/dataWareHouse/BALABALA/02pdw/01_正确数据/21_库存流水/res01
2 ,库存结余,转换格式 :
spark-submit --master yarn --deploy-mode cluster --num-executors 5 --executor-cores 3 --executor-memory 6144m --class lifecycle01_tool.ParseCsvToParquet s3://lifecyclebigdata/dataWareHouse/BALABALA/00jar/02_work/veryOK-1.0-SNAPSHOT.jar lifecyclebigdata/dataWareHouse/BALABALA/01history/2019Q3_3/库存结余.gz lifecyclebigdata/dataWareHouse/BALABALA/02pdw/01_正确数据/21_库存结余/res
3 ,数据查看 :
- 流水 : 查看
spark-submit --master yarn --deploy-mode client --num-executors 5 --executor-cores 3 --executor-memory 6144m --class com.lifecycle.showCount.LookParquet s3://lifecyclebigdata/dataWareHouse/BALABALA/00jar/01_show_count/perfectCode-1.0-SNAPSHOT.jar lifecyclebigdata/dataWareHouse/BALABALA/02pdw/01_正确数据/21_库存流水/res01
- 流水 : 条数
spark-submit --master yarn --deploy-mode client --num-executors 5 --executor-cores 3 --executor-memory 6144m --class com.lifecycle.showCount.CountParquet s3://lifecyclebigdata/dataWareHouse/BALABALA/00jar/01_show_count/perfectCode-1.0-SNAPSHOT.jar lifecyclebigdata/dataWareHouse/BALABALA/02pdw/01_正确数据/21_库存流水/res01
- 结余查看 :
spark-submit --master yarn --deploy-mode client --num-executors 5 --executor-cores 3 --executor-memory 6144m --class com.lifecycle.showCount.LookParquet s3://lifecyclebigdata/dataWareHouse/BALABALA/00jar/01_show_count/perfectCode-1.0-SNAPSHOT.jar lifecyclebigdata/dataWareHouse/BALABALA/02pdw/01_正确数据/22_库存盘点/res
- 结余条数 :
spark-submit --master yarn --deploy-mode client --num-executors 5 --executor-cores 3 --executor-memory 6144m --class com.lifecycle.showCount.CountParquet s3://lifecyclebigdata/dataWareHouse/BALABALA/00jar/01_show_count/perfectCode-1.0-SNAPSHOT.jar lifecyclebigdata/dataWareHouse/BALABALA/02pdw/01_正确数据/22_库存盘点/res
4 ,库存流水 : 686552730 ( 6.8 亿 )
+-----+--------------+--------+----------+--------+--------+-----------+----+----+--------+----------+
|区域 |店主 |门店类型|门店代码 |变动月份|变动日期|货号 |颜色|尺码|变动类型|库存变动量|
+-----+--------------+--------+----------+--------+--------+-----------+----+----+--------+----------+
|2GHH1|呼和浩特零售商|加盟 |2GHH1S0331|201709 |20170921|27704151204|3760|150 |调拨 |2.0 |
|2GHH1|呼和浩特零售商|加盟 |2GHH1S0391|201708 |20170830|27704151204|3760|130 |调拨 |2.0 |
+-----+--------------+--------+----------+--------+--------+-----------+----+----+--------+----------+
5 ,库存结余 :32381927 ( 0.32 亿 )
+----------+--------------------------+--------+----------+--------+--------+-----------+----+----+------+--------+--------+
|区域 |店主 |门店类型|门店代码 |销售月份|销售日期|货号 |颜色|尺码|吊牌价|变动类型|结余库存|
+----------+--------------------------+--------+----------+--------+--------+-----------+----+----+------+--------+--------+
|23GB1A0020|云阳加盟商(非活动) |加盟 |23GB1R0401|201701 |20170101|2081111023 |8701|130 |179.0 |盘点 |-1.0 |
|21BJA |北京子公司 |直营 |21BJAS1211|201701 |20170101|27721170162|0461|120 |49.0 |盘点 |2.0 |
6 ,合并,月聚合,无尺码 :
- 生成数据 :
spark-submit --master yarn --deploy-mode cluster --num-executors 5 --executor-cores 3 --executor-memory 6144m --class lifecycle03_stock.Stock01_GetOneFile s3://lifecyclebigdata/dataWareHouse/BALABALA/00jar/02_work/veryOK-1.0-SNAPSHOT.jar
- 数据样子 :
+----------+----------+--------+---------------+----------+
|区域 |门店代码 |变动月份|款色号 |库存变动量|
+----------+----------+--------+---------------+----------+
|2IWL1 |2IWL1S0761|201910 |244241815384610|5.0 |
|2FQD1A0003|2FQD1R0101|201909 |277041901020366|6.0 |
7 ,库存,商品,门店 :
来源:CSDN
作者:孙砚秋
链接:https://blog.csdn.net/qq_34319644/article/details/103473952