zoukankan      html  css  js  c++  java
  • 在IDEA上用python来连接集群上的hive

    1.在使用Python连接hive之前需要将hive中的文件拷贝到自己创建python项目中

    cp -r apache-hive-0.14.0-bin/lib/py  /home/jia/Desktop

    2.把hive上的py目录下载到桌面之后,进入py目录,复制里面所有的文件到你创建的python项目下

    3.新建一个myHiveLink.py文件,访问hive的代码如下

    import sys
    from hive_service import ThriftHive
    from hive_service.ttypes import HiveServerException
    from thrift import Thrift
    from thrift.transport import TSocket
    from thrift.transport import TTransport
    from thrift.protocol import TBinaryProtocol
    
    def hiveExe(sql):
        try:
            transport = TSocket.TSocket('121.8.xxx.xx', 10000)
            transport = TTransport.TBufferedTransport(transport)
            protocol = TBinaryProtocol.TBinaryProtocol(transport)
            client = ThriftHive.Client(protocol)
            transport.open()
    
            client.execute(sql)
    
            print "The return value is : "
            resultSets=client.fetchAll()
            for j in range(len(resultSets)):
                print resultSets[j]
            print "............"
            transport.close()
        except Thrift.TException, tx:
            print '%s' % (tx.message)
    
            return resultSets
    
    if __name__ == '__main__':
        hiveExe("select * from tableName limit 10")

    4.访问结果如下:

    /usr/bin/python2.7 /home/sendi/IdeaProjects/hive_python/.idea/HiveLink/myHiveLink.py
    The return value is : 
    460030087488272    1333280xxxx    2433    16881    20150608005859    20150608005859    NULL    113.4092361    23.1117361    20150608
    460037560630758    1812682xxxx    2817    5920    20150608005850    20150608005858    16    113.39436    23.42839    20150608
    460030991691721    1532152xxxx    2817    31314    20150608005851    20150608005858    16    113.34354    23.28444    20150608
    460036360705126    1537219xxxx   2817    22386    20150608005851    20150608005858    16    113.3470139    23.2713194    20150608
    460036360703708    1532570xxxx   2817    3617    20150608005853    20150608005858    16    113.3468056    23.3133333    20150608
    460036810304576    1533673xxxx   2817    322    20150608005851    20150608005858    16    113.359375    23.2908333    20150608
    460036110265159    1530011xxxx   2817    3409    20150608005854    20150608005858    16    113.3260417    23.2946528    20150608
    460030991632527    1532152xxxx    2817    22384    20150608005854    20150608005858    16    113.3470139    23.2713194    20150608
    460036671118650    1895716xxxx    2817    31360    20150608005853    20150608005858    16    113.35415    23.30307    20150608
    460036360702214    1534570xxxx   2817    22386    20150608005851    20150608005858    16    113.3470139    23.2713194    20150608

    5.如果访问不了,可能是没有启动hive,服务,则进入hive的bin目录启动服务

    hive --service hiveserver &
  • 相关阅读:
    嵌入式成长轨迹27 【Linux应用编程强化】【中嵌第二阶段】【进程管理】
    嵌入式成长轨迹24【Linux应用编程强化】【Linux下的C编程 下】【实例:Linux命令实现】
    纯CSS代码实现翻页
    Adodb.Stream读取和写入UTF8编码的文件
    对c#拆装箱的性能分析(泛型)
    js自动更换图片代码(收藏)
    提高网站可用性的10个小技巧
    分享下我的家乡语言——湘潭话
    解析用户研究
    HTML5 搭建移动Web应用
  • 原文地址:https://www.cnblogs.com/aijianiula/p/4670010.html
Copyright © 2011-2022 走看看