knowledge_pig

1 Pig运行脚本语法

1.1 mapreduce模式

pig -x mapreduce
-param default_parallel=1
-param rollupreg_extract_jar=hdfs://hdfsenv/tmp/wzhang12/sample-jar-with-dependencies.jar
-param rollupreg_class_name=com.paypal.rollupreg.RollupregExtract
-param hdfs_src_path=hdfs://hdfsenv/tmp/wzhang12/
-param hdfs_dest_path=hdfs://hdfsenv/tmp/wzhang12/dest/
-l /x/home/pp_dt_cmpln_batch/test/extract_pig_20201101.log
-f /etl/LVS/dmetldata06/compliance/bin/extract.pig

/x/home/pp_dt_cmpln_batch/test/extract_pig_20201101_2.log

1.2 local模式

pig -x local
-param default_parallel=1
-param rollupreg_extract_jar=/x/home/pp_dt_cmpln_batch/test/CRR_rollupreg_extract-jar-with-dependencies.jar
-param rollupreg_class_name=com.paypal.rollupreg.RollupregExtract
-param hdfs_src_path=/x/home/pp_dt_cmpln_batch/test/data/
-param hdfs_dest_path=/x/home/pp_dt_cmpln_batch/test/dest/
-param logical_date=20201101
-l /x/home/pp_dt_cmpln_batch/test/extract_pig_20201101.log
-f /x/home/pp_dt_cmpln_batch/test/extract.pig

/x/home/pp_dt_cmpln_batch/test/extract_pig_20201101.log