1. 安装happybase
在conda环境里安装 pip install happybase==1.1.0 -i https://mirrors.aliyun.com/pypi/simple/
报错:
Failed to build thriftpy
Installing collected packages: thriftpy, happybase
Running setup.py install for thriftpy ... error
ERROR: Command errored out with exit status 1:
command: /opt/module/miniconda3/envs/reco_sys/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-3hnoho7s/thriftpy_d7e1467ce8a74e82a8b8e82d3e942202/setup.py'"'"'; __file__='"'"'/tmp/pip-install-3hnoho7s/thriftpy_d7e1467ce8a74e82a8b8e82d3e942202/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-g8qrieks/install-record.txt --single-version-externally-managed --compile --install-headers /opt/module/miniconda3/envs/reco_sys/include/python3.6m/thriftpy
cwd: /tmp/pip-install-3hnoho7s/thriftpy_d7e1467ce8a74e82a8b8e82d3e942202/
Complete output (51 lines):
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/thriftpy
copying thriftpy/__init__.py -> build/lib.linux-x86_64-3.6/thriftpy
copying thriftpy/_compat.py -> build/lib.linux-x86_64-3.6/thriftpy
copying thriftpy/hook.py -> build/lib.linux-x86_64-3.6/thriftpy
copying thriftpy/rpc.py -> build/lib.linux-x86_64-3.6/thriftpy
copying thriftpy/server.py -> build/lib.linux-x86_64-3.6/thriftpy
copying thriftpy/thrift.py -> build/lib.linux-x86_64-3.6/thriftpy
copying thriftpy/tornado.py -> build/lib.linux-x86_64-3.6/thriftpy
copying thriftpy/utils.py -> build/lib.linux-x86_64-3.6/thriftpy
creating build/lib.linux-x86_64-3.6/thriftpy/contrib
copying thriftpy/contrib/__init__.py -> build/lib.linux-x86_64-3.6/thriftpy/contrib
creating build/lib.linux-x86_64-3.6/thriftpy/parser
copying thriftpy/parser/__init__.py -> build/lib.linux-x86_64-3.6/thriftpy/parser
copying thriftpy/parser/exc.py -> build/lib.linux-x86_64-3.6/thriftpy/parser
copying thriftpy/parser/lexer.py -> build/lib.linux-x86_64-3.6/thriftpy/parser
copying thriftpy/parser/parser.py -> build/lib.linux-x86_64-3.6/thriftpy/parser
creating build/lib.linux-x86_64-3.6/thriftpy/protocol
copying thriftpy/protocol/__init__.py -> build/lib.linux-x86_64-3.6/thriftpy/protocol
copying thriftpy/protocol/binary.py -> build/lib.linux-x86_64-3.6/thriftpy/protocol
copying thriftpy/protocol/compact.py -> build/lib.linux-x86_64-3.6/thriftpy/protocol
copying thriftpy/protocol/exc.py -> build/lib.linux-x86_64-3.6/thriftpy/protocol
copying thriftpy/protocol/json.py -> build/lib.linux-x86_64-3.6/thriftpy/protocol
copying thriftpy/protocol/multiplex.py -> build/lib.linux-x86_64-3.6/thriftpy/protocol
creating build/lib.linux-x86_64-3.6/thriftpy/transport
copying thriftpy/transport/__init__.py -> build/lib.linux-x86_64-3.6/thriftpy/transport
copying thriftpy/transport/_ssl.py -> build/lib.linux-x86_64-3.6/thriftpy/transport
copying thriftpy/transport/socket.py -> build/lib.linux-x86_64-3.6/thriftpy/transport
copying thriftpy/transport/sslsocket.py -> build/lib.linux-x86_64-3.6/thriftpy/transport
creating build/lib.linux-x86_64-3.6/thriftpy/contrib/tracking
copying thriftpy/contrib/tracking/__init__.py -> build/lib.linux-x86_64-3.6/thriftpy/contrib/tracking
copying thriftpy/contrib/tracking/tracker.py -> build/lib.linux-x86_64-3.6/thriftpy/contrib/tracking
creating build/lib.linux-x86_64-3.6/thriftpy/transport/buffered
copying thriftpy/transport/buffered/__init__.py -> build/lib.linux-x86_64-3.6/thriftpy/transport/buffered
creating build/lib.linux-x86_64-3.6/thriftpy/transport/framed
copying thriftpy/transport/framed/__init__.py -> build/lib.linux-x86_64-3.6/thriftpy/transport/framed
creating build/lib.linux-x86_64-3.6/thriftpy/transport/memory
copying thriftpy/transport/memory/__init__.py -> build/lib.linux-x86_64-3.6/thriftpy/transport/memory
copying thriftpy/contrib/tracking/tracking.thrift -> build/lib.linux-x86_64-3.6/thriftpy/contrib/tracking
running build_ext
building 'thriftpy.transport.cybase' extension
creating build/temp.linux-x86_64-3.6
creating build/temp.linux-x86_64-3.6/thriftpy
creating build/temp.linux-x86_64-3.6/thriftpy/transport
gcc -pthread -B /opt/module/miniconda3/envs/reco_sys/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/opt/module/miniconda3/envs/reco_sys/include/python3.6m -c thriftpy/transport/cybase.c -o build/temp.linux-x86_64-3.6/thriftpy/transport/cybase.o
unable to execute 'gcc': No such file or directory
error: command 'gcc' failed with exit status 1
----------------------------------------
ERROR: Command errored out with exit status 1: /opt/module/miniconda3/envs/reco_sys/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-3hnoho7s/thriftpy_d7e1467ce8a74e82a8b8e82d3e942202/setup.py'"'"'; __file__='"'"'/tmp/pip-install-3hnoho7s/thriftpy_d7e1467ce8a74e82a8b8e82d3e942202/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-g8qrieks/install-record.txt --single-version-externally-managed --compile --install-headers /opt/module/miniconda3/envs/reco_sys/include/python3.6m/thriftpy Check the logs for full command output.
大致意思是缺少了C++的环境,于是乎各种搜索之后,另一条命令安排上yum install libboost-dev libboost-test-dev libboost-program-options-dev libevent-dev automake libtool flex bison pkg-config g++ libssl-dev
,期间也是漫长的等待,没有科学上网的多试几次总有一次成功。
接下来的pip install happybase
就成功了。
2. thrift
要想要happybase操作hbase,还得启动hbase thrift server:hbase-daemon.sh start thrift
3. 建立连接
hostname = 'hadoop-master'
table_name = 'users'
column_family = 'cf'
conn = happybase.Connection(hostname)
print(conn.tables())
至此大功告成。