Python操作Hbase

目录：

1.原理介绍

2.安装环境所依赖的工具

3.安装Thrift

4.拷贝python操作hbase 符合Thrift协议的API目录到开发目录

5.启动Thrift服务

6.启动Hbase

7.测试

 

1.原理介绍

 

 

2.安装环境所依赖的工具

sudo yum install automake libtool flex bison pkgconfig gcc-c++ boost-devel libevent-devel zlib-devel python-devel ruby-devel openssl-devel
sudo yum install boost-devel.x86_64
sudo yum install libevent-devel.x86_64

 

3.安装Thrift

./configure --with-cpp=no --with-ruby=no
make
sudo make install     (若不是以root用户执行make install，普通用户要加sudo，不然不能在/usr/local/bin下创建thrift可执行文件，会出现下面的错误)

 

4.产生针对Python的Hbase的API

　　1.查找 Hbase.thrift ，此文件是python针对Hbase操作的符合Thrift协议的接口。

[liangjf@master hbase-0.98.0]$ find . -name Hbase.thrift
  ./hbase-thrift/src/main/resources/org/apache/hadoop/hbase/thrift/Hbase.thrift
  [liangjf@master hbase-0.98.0]$ cd ./hbase-thrift/src/main/resources/org/apache/hadoop/hbase/thrift/
  [liangjf@master thrift]$ ls
  Hbase.thrift

 

 　　2.产生针对Python的Hbase的API

thrift -gen py Hbase.thrif

3.拷贝到python_hbase的开发目录，把此目录里面的接口文件相当于模块调用，这里用到的是python的__init__.py的知识，请自行查看。

cp -raf gen-py/hbase/ /home/liangjf/big_data/hbase/python_hbase   【python_hbase，修改为此名字，方便辨别。后面开发导入时要注意名字

若步骤2 安装 Thrift 时，没有用sudo make install ，不会产生thrift可执行文件的。

[liangjf@master thrift]$ thrift -gen py Hbase.thrift
-bash: thrift: command not found

重新安装发现这里有错误。错误明显是没权限在/usr/local/bin目录创建可执行文件thrift。因此在终端不能直接thrift。所以要这样：sudo make install

[liangjf@master thrift-0.8.0]$ make install
Making install in compiler/cpp
make[1]: Entering directory `/home/liangjf/app/thrift-0.8.0/compiler/cpp'
make  install-am
make[2]: Entering directory `/home/liangjf/app/thrift-0.8.0/compiler/cpp'
make[3]: Entering directory `/home/liangjf/app/thrift-0.8.0/compiler/cpp'
test -z "/usr/local/bin" || /bin/mkdir -p "/usr/local/bin"
  /bin/sh ../../libtool   --mode=install /usr/bin/install -c thrift '/usr/local/bin'
libtool: install: /usr/bin/install -c thrift /usr/local/bin/thrift
/usr/bin/install: cannot create regular file `/usr/local/bin/thrift': Permission denied
make[3]: *** [install-binPROGRAMS] Error 1
make[3]: Leaving directory `/home/liangjf/app/thrift-0.8.0/compiler/cpp'
make[2]: *** [install-am] Error 2
make[2]: Leaving directory `/home/liangjf/app/thrift-0.8.0/compiler/cpp'
make[1]: *** [install] Error 2
make[1]: Leaving directory `/home/liangjf/app/thrift-0.8.0/compiler/cpp'
make: *** [install-recursive] Error 1

 

5.启动Thrift服务，在hbase/bin目录下

hbase-daemon.sh start thrift

 

6.启动Hbase，在hbase/bin目录下

./start-hbase.sh

 

 

7.测试

【create_table.py】--------------建表

from thrift import Thrift
from thrift.transport import TSocket
from thrift.transport import TTransport
from thrift.protocol import TBinaryProtocol
from pyhon_hbase import Hbase
from pyhon_hbase.ttypes import *
transport = TSocket.TSocket('master', 9090);
transport = TTransport.TBufferedTransport(transport)
protocol = TBinaryProtocol.TBinaryProtocol(transport);
client = Hbase.Client(protocol)
transport.open()
contents = ColumnDescriptor(name='cf:', maxVersions=1)
client.createTable('test', [contents])
------------------------
hbase(main):023:0> list
TABLE                                                                                                                                                    
member                                                                                                                                                   
test                                                                                                                                                     
2 row(s) in 0.0150 seconds
=> ["member", "test"]

-------------------------------------------------------------------

【insert_data.py】--------------插入数据

from thrift import Thrift
from thrift.transport import TSocket
from thrift.transport import TTransport
from thrift.protocol import TBinaryProtocol
from pyhon_hbase import Hbase
from pyhon_hbase.ttypes import *
transport = TSocket.TSocket('master', 9090)
transport = TTransport.TBufferedTransport(transport)
protocol = TBinaryProtocol.TBinaryProtocol(transport)
client = Hbase.Client(protocol)
transport.open()
row = 'id-1'
mutations = [Mutation(column="cf:name", value="liangjf")]
client.mutateRow('test', row, mutations, None)
------------------------
hbase(main):022:0> scan 'test'
ROW                                     COLUMN+CELL                                                                                                      
id-1                                   column=cf:name, timestamp=1511079251051, value=liangjf
1 row(s) in 0.0130 seconds

-------------------------------------------------------------------

【getRow.py】--------------获取一行记录

from thrift import Thrift
from thrift.transport import TSocket
from thrift.transport import TTransport
from thrift.protocol import TBinaryProtocol
from python_hbase import Hbase
from python_hbase.ttypes import *
transport = TSocket.TSocket('master', 9090)
transport = TTransport.TBufferedTransport(transport)
protocol = TBinaryProtocol.TBinaryProtocol(transport)
client = Hbase.Client(protocol)
transport.open()
tableName = 'test'
rowKey = 'id-1'
result = client.getRow(tableName, rowKey, None)
print result
for r in result:
    print 'the row is ' , r.row
    print 'the values is ' , r.columns.get('cf:name').value
----------------------
[liangjf@master python_hbase]$ python getRow.py
[TRowResult(sortedColumns=None, columns={'cf:name': TCell(timestamp=1511079251051, value='liangjf')}, row='id-1')]
the row is  id-1
the values is  liangjf