zoukankan      html  css  js  c++  java
  • pig笔记

    1.安装Pig
    将pig添加到环境变量当中

    2.pig使用
    首先将数据库中的数据导入到HDFS上
    sqoop import --connect jdbc:mysql://192.168.1.10:3306/cloud --username root --


    password JChubby123  --table trade_detail --target-dir '/sqoop/td'
    sqoop import --connect jdbc:mysql://192.168.1.10:3306/cloud --username root --


    password JChubby123  --table user_info --target-dir '/sqoop/ui'
     

    td = load '/sqoop/td' using PigStorage(',') as (id:long, account:chararray, 


    income:double, expenses:double, time:chararray);
    ui = load '/sqoop/ui' using PigStorage(',') as (id:long, account:chararray, 


    name:chararray, age:int);

    td1 = foreach td generate account, income, expenses, income-expenses as surplus;

    td2 = group td1 by account;

    td3 = foreach td2 generate group as account, SUM(td1.income) as income, SUM


    (td1.expenses) as expenses, SUM(td1.surplus) as surplus;

    tu = join td3 by account, ui by account;

    result = foreach tu generate td3::account as account, ui::name, td3::income, 


    td3::expenses, td3::surplus;

    store result into '/result' using PigStorage(',');
  • 相关阅读:
    Python脚本文件(.py)打包为可执行文件(.exe)即避免命令行中包含Python解释器
    MVC 三步完成增删改查设计
    MVC中使用SqlServerCe
    回车转Tab
    动态代码
    Mvc 用户没有登录跳转到登录界面
    Mvc
    Mvc提交
    EF查询 linq
    EF数据迁移 Migrations
  • 原文地址:https://www.cnblogs.com/jchubby/p/4429688.html
Copyright © 2011-2022 走看看