python通过hbase的client读取数据

使用的python版本为3.5.2，使用过程中发现了好几个坑

首先安装thrift和hbase-thrift

pip install thrift
pip install hbase-thrift

安装完成之后第一次运行，报错误：

in <module> from hbase import Hbase File 
"C:\Users\tianxiao\AppData\Local\Programs\Python\Python36\lib\site-packages\hbase\Hbase.py", line 2066 
except IOError, io: ^ SyntaxError: invalid syntax

原因是python版本带来的语法兼容性问题，下载python3的Hbase文件，替换Hbase文件/usr/local/lib/python3.6/dist-packages/hbase/Hbase.py和ttypes.py
下载地址为:https://github.com/626626cdllp/infrastructure/tree/master/hbase

这样，第一个问题就解决了。

然后用如下代码：

from thrift.transport import TSocket,TTransport
from thrift.protocol import TBinaryProtocol
from hbase import Hbase

# thrift默认端口是9090
socket = TSocket.TSocket('192.1.1.1',9090)
socket.setTimeout(5000)

transport = TTransport.TBufferedTransport(socket)
protocol = TBinaryProtocol.TBinaryProtocol(transport)

client = Hbase.Client(protocol)
socket.open()

print(client.getTableNames())
print(client.get('test','row1','cf:a'))

报另一个错误：thrift.transport.TTransport.TTransportException: TSocket read 0 bytes，详细报错日志为：

Traceback (most recent call last):
  File "...../test2.py", line 69, in <module>
    print(client.getTableNames())
  File "C:\Program Files (x86)\python3.5\lib\site-packages\hbase\Hbase.py", line 738, in getTableNames
    return self.recv_getTableNames()
  File "C:\Program Files (x86)\python3.5\lib\site-packages\hbase\Hbase.py", line 748, in recv_getTableNames
    (fname, mtype, rseqid) = self._iprot.readMessageBegin()
  File "C:\Program Files (x86)\python3.5\lib\site-packages\thrift\protocol\TBinaryProtocol.py", line 134, in readMessageBegin
    sz = self.readI32()
  File "C:\Program Files (x86)\python3.5\lib\site-packages\thrift\protocol\TBinaryProtocol.py", line 217, in readI32
    buff = self.trans.readAll(4)
  File "C:\Program Files (x86)\python3.5\lib\site-packages\thrift\transport\TTransport.py", line 61, in readAll
    chunk = self.read(sz - have)
  File "C:\Program Files (x86)\python3.5\lib\site-packages\thrift\transport\TTransport.py", line 163, in read
    self.__rbuf = BufferIO(self.__trans.read(max(sz, self.__rbuf_size)))
  File "C:\Program Files (x86)\python3.5\lib\site-packages\thrift\transport\TSocket.py", line 132, in read
    message='TSocket read 0 bytes')
thrift.transport.TTransport.TTransportException: TSocket read 0 bytes

原因是thrift 的server端和client端的协议不匹配造成的，python要使用TCompactProtocol，而不能使用TBinaryProtocol，

因此修改代码为：

from thrift.transport import TSocket
from thrift.transport.TTransport import TFramedTransport
from thrift.protocol import TCompactProtocol
from hbase import Hbase

# thrift默认端口是9090
socket = TSocket.TSocket('10.1.1.1',9090)
socket.setTimeout(5000)

transport = TFramedTransport(socket)
protocol = TCompactProtocol.TCompactProtocol(transport)
client = Hbase.Client(protocol)
transport.open()

# print(client.getTableNames())
print(client.get('dev_user_tags','0000f00437f8','basic_info:age'))

代码可以正常运行。

最后一个问题，如果找不到TFramedTransport和TCompactProtocol，检查site-packages/thrift目录下有没有相关文件，没有，可以在thrift官网下载源码包，并用其中的lib/py/src/覆盖site-packages/thrift/目录即可。