Ubuntu下Python访问Hive——thrift安装及测试

1、下载thrift：

1. wget http://mirror.bjtu.edu.cn/apache//thrift/0.8.0/thrift-0.9.0.tar.gz 

2、安装依赖：

1. sudo apt-get install build-essential   
2. sudo apt-get install bison flex   
3. sudo apt-get install libboost-dev python-dev   
4. sudo apt-get install autoconf automake libtool pkg-config   
5. sudo apt-get install git   

3、安装thrift：

1. tar -xzvf thrift-0.9.0.tar.gz 
2. cd thrift-0.9.0
3. sudo ./configure --prefix=/host/java/thrift
4. sudo make && sudo make install

4、启动Hive Thrift Server  (>>hive --service  hiveserver)

要先启动hadoop 设置hdfs安全模式为leave

>>start-all.sh

>>hadoop dfsadmin -safemode leave

启动时报错……没关系，运行python时候才会再有反应

Starting Hive Thrift Server
13/07/29 13:24:13 WARN conf.HiveConf: DEPRECATED: Configuration property hive.metastore.local no longer has any effect. Make sure to provide a valid value for hive.metastore.uris if you are connecting to a remote metastore.

解决办法：

因为在0.10里面，hive.metastore.local参数已经取消，都使用hive.metastore.uris参数，如果hive.metastore.uris为空，就默认为local模式。

把hive.metastore.local参数删掉就好了。

5、测试代码编写如下：

testHive.py:

#!/usr/bin/env python

import sys

sys.path.append('/usr/local/hadoop/hive/lib/py')#'/usr/local/hadoop/hive是你的hive安装路径

from hive_service import ThriftHive

from hive_service.ttypes import HiveServerException

from thrift import Thrift

from thrift.transport import TSocket

from thrift.transport import TTransport

from thrift.protocol import TBinaryProtocol

 

def hiveExe(sql):

 

    try:

        transport = TSocket.TSocket('127.0.0.1', 10000) 

        transport = TTransport.TBufferedTransport(transport)

        protocol = TBinaryProtocol.TBinaryProtocol(transport)

        client = ThriftHive.Client(protocol)

        transport.open()

 

 

        client.execute(sql)

 

        print "The return value is : " 

        print client.fetchAll()

        print "............"

        transport.close()

    except Thrift.TException, tx:

        print '%s' % (tx.message)

 

if __name__ == '__main__':

    hiveExe("select * from testParttion")#t_afan_test是你的表名

6、执行

>>python testHive.py

7、结果

The return value is:

['[12,23,23,34]\t["what","are","this"]','[34,45,34,23,12]\t["who","am","i","are"]']

转载于:https://www.cnblogs.com/skiller-s/p/3222747.html