记录happybase操作hbase的坑

1. 安装happybase

在conda环境里安装 pip install happybase==1.1.0 -i https://mirrors.aliyun.com/pypi/simple/ 报错：

Failed to build thriftpy
Installing collected packages: thriftpy, happybase
    Running setup.py install for thriftpy ... error
    ERROR: Command errored out with exit status 1:
     command: /opt/module/miniconda3/envs/reco_sys/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-3hnoho7s/thriftpy_d7e1467ce8a74e82a8b8e82d3e942202/setup.py'"'"'; __file__='"'"'/tmp/pip-install-3hnoho7s/thriftpy_d7e1467ce8a74e82a8b8e82d3e942202/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-g8qrieks/install-record.txt --single-version-externally-managed --compile --install-headers /opt/module/miniconda3/envs/reco_sys/include/python3.6m/thriftpy
         cwd: /tmp/pip-install-3hnoho7s/thriftpy_d7e1467ce8a74e82a8b8e82d3e942202/
    Complete output (51 lines):
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.6
    creating build/lib.linux-x86_64-3.6/thriftpy
    copying thriftpy/__init__.py -> build/lib.linux-x86_64-3.6/thriftpy
    copying thriftpy/_compat.py -> build/lib.linux-x86_64-3.6/thriftpy
    copying thriftpy/hook.py -> build/lib.linux-x86_64-3.6/thriftpy
    copying thriftpy/rpc.py -> build/lib.linux-x86_64-3.6/thriftpy
    copying thriftpy/server.py -> build/lib.linux-x86_64-3.6/thriftpy
    copying thriftpy/thrift.py -> build/lib.linux-x86_64-3.6/thriftpy
    copying thriftpy/tornado.py -> build/lib.linux-x86_64-3.6/thriftpy
    copying thriftpy/utils.py -> build/lib.linux-x86_64-3.6/thriftpy
    creating build/lib.linux-x86_64-3.6/thriftpy/contrib
    copying thriftpy/contrib/__init__.py -> build/lib.linux-x86_64-3.6/thriftpy/contrib
    creating build/lib.linux-x86_64-3.6/thriftpy/parser
    copying thriftpy/parser/__init__.py -> build/lib.linux-x86_64-3.6/thriftpy/parser
    copying thriftpy/parser/exc.py -> build/lib.linux-x86_64-3.6/thriftpy/parser
    copying thriftpy/parser/lexer.py -> build/lib.linux-x86_64-3.6/thriftpy/parser
    copying thriftpy/parser/parser.py -> build/lib.linux-x86_64-3.6/thriftpy/parser
    creating build/lib.linux-x86_64-3.6/thriftpy/protocol
    copying thriftpy/protocol/__init__.py -> build/lib.linux-x86_64-3.6/thriftpy/protocol
    copying thriftpy/protocol/binary.py -> build/lib.linux-x86_64-3.6/thriftpy/protocol
    copying thriftpy/protocol/compact.py -> build/lib.linux-x86_64-3.6/thriftpy/protocol
    copying thriftpy/protocol/exc.py -> build/lib.linux-x86_64-3.6/thriftpy/protocol
    copying thriftpy/protocol/json.py -> build/lib.linux-x86_64-3.6/thriftpy/protocol
    copying thriftpy/protocol/multiplex.py -> build/lib.linux-x86_64-3.6/thriftpy/protocol
    creating build/lib.linux-x86_64-3.6/thriftpy/transport
    copying thriftpy/transport/__init__.py -> build/lib.linux-x86_64-3.6/thriftpy/transport
    copying thriftpy/transport/_ssl.py -> build/lib.linux-x86_64-3.6/thriftpy/transport
    copying thriftpy/transport/socket.py -> build/lib.linux-x86_64-3.6/thriftpy/transport
    copying thriftpy/transport/sslsocket.py -> build/lib.linux-x86_64-3.6/thriftpy/transport
    creating build/lib.linux-x86_64-3.6/thriftpy/contrib/tracking
    copying thriftpy/contrib/tracking/__init__.py -> build/lib.linux-x86_64-3.6/thriftpy/contrib/tracking
    copying thriftpy/contrib/tracking/tracker.py -> build/lib.linux-x86_64-3.6/thriftpy/contrib/tracking
    creating build/lib.linux-x86_64-3.6/thriftpy/transport/buffered
    copying thriftpy/transport/buffered/__init__.py -> build/lib.linux-x86_64-3.6/thriftpy/transport/buffered
    creating build/lib.linux-x86_64-3.6/thriftpy/transport/framed
    copying thriftpy/transport/framed/__init__.py -> build/lib.linux-x86_64-3.6/thriftpy/transport/framed
    creating build/lib.linux-x86_64-3.6/thriftpy/transport/memory
    copying thriftpy/transport/memory/__init__.py -> build/lib.linux-x86_64-3.6/thriftpy/transport/memory
    copying thriftpy/contrib/tracking/tracking.thrift -> build/lib.linux-x86_64-3.6/thriftpy/contrib/tracking
    running build_ext
    building 'thriftpy.transport.cybase' extension
    creating build/temp.linux-x86_64-3.6
    creating build/temp.linux-x86_64-3.6/thriftpy
    creating build/temp.linux-x86_64-3.6/thriftpy/transport
    gcc -pthread -B /opt/module/miniconda3/envs/reco_sys/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/opt/module/miniconda3/envs/reco_sys/include/python3.6m -c thriftpy/transport/cybase.c -o build/temp.linux-x86_64-3.6/thriftpy/transport/cybase.o
    unable to execute 'gcc': No such file or directory
    error: command 'gcc' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /opt/module/miniconda3/envs/reco_sys/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-3hnoho7s/thriftpy_d7e1467ce8a74e82a8b8e82d3e942202/setup.py'"'"'; __file__='"'"'/tmp/pip-install-3hnoho7s/thriftpy_d7e1467ce8a74e82a8b8e82d3e942202/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-g8qrieks/install-record.txt --single-version-externally-managed --compile --install-headers /opt/module/miniconda3/envs/reco_sys/include/python3.6m/thriftpy Check the logs for full command output.

大致意思是缺少了C++的环境，于是乎各种搜索之后，另一条命令安排上yum install libboost-dev libboost-test-dev libboost-program-options-dev libevent-dev automake libtool flex bison pkg-config g++ libssl-dev，期间也是漫长的等待，没有科学上网的多试几次总有一次成功。

接下来的pip install happybase就成功了。

2. thrift

要想要happybase操作hbase，还得启动hbase thrift server：hbase-daemon.sh start thrift

3. 建立连接

hostname = 'hadoop-master'
table_name = 'users'
column_family = 'cf'


conn = happybase.Connection(hostname)

print(conn.tables())

至此大功告成。