zoukankan      html  css  js  c++  java
  • CentOS6.5下通过Thrift使用Python连接操作hive 安装配置记录

    参考博客:http://blog.csdn.net/jiedushi/article/details/7531722

    1. 安装Thrift

    1.安装依赖库

    yum install automake libtool flex bison pkgconfig gcc-c++ boost-devel libevent-devel zlib-devel python-devel ruby-devel 
    
    1. 下载链接thrift-0.10.0
    1. 解压安装
    tar xvf thrift-0.10.0.tar.gz
    cd thrift-0.10.0
    ./configure
    make
    sudo make install
    

    ./configure报错configure: error: Bison version 2.5 or higher must be installed on the system!
    yum的base源默认安装的bison版本为2.4.1太低,下载bison-2.5安装包编译安装

     wget http://ftp.gnu.org/gnu/bison/bison-2.5.1.tar.gz tar xvf  tar xvf
     bison-2.5.1.tar.gz  cd bison-2.5.1 ./configure make sudo make install
    

    回到thrift文件夹下重新执行./configure

    2. 复制Hive文件

    cp -r $HIVE_PATH/lib/py/* /usr/local/lib/python2.7/site-packages

    3. 启动hiveserver2

    hiveserver2 &

    4. Python中编程

    Hive官网的示例代码:https://cwiki.apache.org/confluence/display/Hive/HiveClient#HiveClient-Python

    import sys
    from hive_service import ThriftHive
    from hive_service.ttypes import HiveServerException
    from thrift import Thrift
    from thrift.transport import TSocket
    from thrift.transport import TTransport
    from thrift.protocol import TBinaryProtocol
     
    try:
        transport = TSocket.TSocket('localhost', 10000)
        transport = TTransport.TBufferedTransport(transport)
        protocol = TBinaryProtocol.TBinaryProtocol(transport)
     
        client = ThriftHive.Client(protocol)
        transport.open()
     
        client.execute("CREATE TABLE r(a STRING, b INT, c DOUBLE)")
        client.execute("LOAD TABLE LOCAL INPATH '/path' INTO TABLE r")
        client.execute("SELECT * FROM r")
        while (1):
          row = client.fetchOne()
          if (row == None):
            break
          print row
        client.execute("SELECT * FROM r")
        print client.fetchAll()
     
        transport.close()
     
    except Thrift.TException, tx:
        print '%s' % (tx.message)
    

    不过收不到响应,问题待解决

  • 相关阅读:
    希尔排序之C++实现(初级版)
    CF9D How many trees?
    IOI2015 boxes纪念品盒
    CSP-S 2019图论总结
    数据生成器
    Special-Judge模板
    CF293B Distinct Paths
    浅谈几种常见的剪枝方式
    CF620E New Year Tree
    浅谈DFS序
  • 原文地址:https://www.cnblogs.com/KattyJoy/p/6540125.html
Copyright © 2011-2022 走看看