kafka python使用方法_kfka学习笔记一：使用Python操作Kafka

1、准备工作

使用python操作kafka目前比较常用的库是kafka-python库，但是在安装这个库的时候需要依赖setuptools库和six库，下面就要分别来下载这几个库

https://pypi.org/project/kafka-python/#files

1.1 下载setuptools

1.2 下载kafka-python

1.3 下载six

下载略

注意版本：

我开始用的setuptools-0.6c11-py2.6.egg + kafka-python-1.4.4.tar.gz + six-1.12.0.tar.gz包，安装后，from kafka import KafkaProducer 是出错，可能原因事版本问题。重新换了一下面的版本，就好了。

2、安装相关python库

在上一步里面我们已经下载了好相关的包，下面开始具体安装，首先创建一个/opt/python_library/，然后把这几个包文件上传到这里

[root@centos03 python_library]# ll

total 600

drwxr-xr-x. 7 5579 5579   4096 Dec  1 21:20 kafka-python-1.3.5

-rw-r--r--. 1 root root 227497 Dec  1 21:18 kafka-python-1.3.5.tar.gz

-rw-r--r--. 1 root root 333447 Dec  1 20:26 setuptools-0.6c11-py2.6.egg

drwxr-xr-x. 6 2000 2000   4096 Dec  1 21:19 six-1.11.0

-rw-r--r--. 1 root root  29860 Dec  1 21:18 six-1.11.0.tar.gz

[root@centos03 python_library]#

2.1 安装setuptools

[root@centos03 python_library]# sh setuptools-0.6c11-py2.6.egg

Processing setuptools-0.6c11-py2.6.egg

Copying setuptools-0.6c11-py2.6.egg to /usr/lib/python2.6/site-packages

Adding setuptools0.6c11 to easy-install.pth file

Installing easy_install script to/usr/bin

Installing easy_install-2.6 script to /usr/bin

Installed/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg

Processing dependenciesfor setuptools==0.6c11

Finished processing dependenciesfor setuptools==0.6c11

[root@centos03 python_library]#

2.2  安装six

[root@centos03 six-1.11.0]# ll

total 140

drwxr-xr-x. 4 root root  4096 Dec  1 21:19 build

-rw-r--r--. 1 2000 2000  7970 Sep 18  2017 CHANGES

drwxr-xr-x. 2 root root  4096 Dec  1 21:19 dist

drwxr-xr-x. 2 2000 2000  4096 Sep 18  2017 documentation

-rw-r--r--. 1 2000 2000  1066 Sep 18  2017 LICENSE

-rw-r--r--. 1 2000 2000   114 Sep 18  2017 MANIFEST.in

-rw-r--r--. 1 2000 2000  1818 Sep 18  2017 PKG-INFO

-rw-r--r--. 1 2000 2000  1096 Sep 18  2017 README.rst

-rw-r--r--. 1 2000 2000   282 Sep 18  2017 setup.cfg

-rw-r--r--. 1 2000 2000  2184 Sep 18  2017 setup.py

drwxr-xr-x. 2 2000 2000  4096 Sep 18  2017 six.egg-info

-rw-r--r--. 1 2000 2000 30888 Sep 18  2017 six.py

-rw-r--r--. 1 root root 30782 Dec  1 21:19 six.pyc

-rw-r--r--. 1 2000 2000 26864 Sep 18  2017 test_six.py

[root@centos03 six-1.11.0]#

执行python setup.py install

[root@centos03 six-1.11.0]# python setup.py install

running install

running bdist_egg

running egg_info

writing six.egg-info/PKG-INFO

writing top-level names to six.egg-info/top_level.txt

writing dependency_links to six.egg-info/dependency_links.txt

writing six.egg-info/PKG-INFO

writing top-level names to six.egg-info/top_level.txt

writing dependency_links to six.egg-info/dependency_links.txt

reading manifest file 'six.egg-info/SOURCES.txt'

reading manifest template 'MANIFEST.in'

no previously-included directories found matching 'documentation/_build'

writing manifest file 'six.egg-info/SOURCES.txt'

installing library code to build/bdist.linux-x86_64/egg

running install_lib

running build_py

creating build/bdist.linux-x86_64/egg

copying build/lib/six.py -> build/bdist.linux-x86_64/egg

byte-compiling build/bdist.linux-x86_64/egg/six.py to six.pyc

creating build/bdist.linux-x86_64/egg/EGG-INFO

copying six.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO

copying six.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO

copying six.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO

copying six.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO

zip_safe flag not set; analyzing archive contents...

six: module references __path__

creating 'dist/six-1.11.0-py2.6.egg' and adding 'build/bdist.linux-x86_64/egg' to it

removing 'build/bdist.linux-x86_64/egg' (and everything under it)

Processing six-1.11.0-py2.6.egg

removing '/usr/lib/python2.6/site-packages/six-1.11.0-py2.6.egg' (and everything under it)

creating /usr/lib/python2.6/site-packages/six-1.11.0-py2.6.egg

Extracting six-1.11.0-py2.6.egg to /usr/lib/python2.6/site-packages

six 1.11.0 is already the active version in easy-install.pth

Installed /usr/lib/python2.6/site-packages/six-1.11.0-py2.6.egg

Processing dependencies for six==1.11.0

Finished processing dependencies for six==1.11.0

[root@centos03 six-1.11.0]#

2.3 安装kafka-python

执行tar -zxvf kafka-python-1.3.4.tar.gz解压安装包，会产生kafka-python-1.3.4文件夹，进入到该文件夹

[root@centos03 python_library]# tar -zxvf kafka-python-1.3.5.tar.gz

[root@centos03 kafka-python-1.3.5]# ll

total 100

-rw-r--r--.  1 5579 5579  2508 Jan 24  2016 AUTHORS.md

drwxr-xr