zoukankan      html  css  js  c++  java
  • 使用impala连接hive踩坑过程

    一、打包镜像出错

     docker build总是出错,如果你用的是python3.7,可以考虑使用python3.6版本

     并且注意:选择thrift-sasl==0.2.1,否则会出现:

    AttributeError: 'TSocket' object has no attribute 'isOpen'

    二、auth_mechanism

    from impala.dbapi import connect
    
    host='your_hive_ip'
    username='your_username'
    password='your_password'
    port=21050
    data_base_name='your_database_name'
    
    db_connection = connect(host=host, port=port, user=username, password=password, database=data_base_name, auth_mechanism='LDAP')
    # 首先不加auth_mechanism时,该行不会报错,但是下面的cursor会报错
    # 加上auth_mechanism参数,会出现下面的错误
    cursor = db_connection.cursor()
    thriftpy.transport.TTransportException: TTransportException(type=1, message="Could not start SASL: b'Error in sasl_client_start (-4) SASL(-4): no mechanism available: Unable to find a callback: 2'")
    

      主要原因其实还是因为sasl和pure-sasl有冲突,这种情况下,直接卸载sasl包,然后安装puresasl,解决方案来自git上:

    I solved the issue, had to uninstall the package SASL and install PURE-SASL, when impyla can´t find the sasl package it works with pure-sasl and then everything goes well.
    

      也可以直接下载puresasl包到本地,然后pip install

    三、TypeError: can't concat str to bytes

     定位到错误的最后一条,在init.py第94行(标黄的部分):

    header = struct.pack(">BI", status, len(body))
    #按照网上的提供的办法增加对BODY的处理
    if (type(body) is str):
        body = body.encode()
    self._trans.write(header + body)
    self._trans.flush()
    

     最后给出各个包的版本:

      python3.6.4,bitarray==1.1.0,thrift==0.9.3,thrift-sasl==0.2.1,six==1.12.0,pure-sasl==0.6.2,impyla==0.15.0

     更多踩坑过程,请参考

  • 相关阅读:
    POJ 1811 Prime Test 素性测试 分解素因子
    sysbench的安装与使用
    电脑中已有VS2005和VS2010安装.NET3.5失败的解决方案
    I.MX6 show battery states in commandLine
    RPi 2B Raspbian system install
    I.MX6 bq27441 driver porting
    I.MX6 隐藏电池图标
    I.MX6 Power off register hacking
    I.MX6 Goodix GT9xx touchscreen driver porting
    busybox filesystem httpd php-5.5.31 sqlite3 webserver
  • 原文地址:https://www.cnblogs.com/always-fight/p/11887808.html
Copyright © 2011-2022 走看看