kfka学习笔记二：使用Python操作Kafka

1、准备工作

使用python操作kafka目前比较常用的库是kafka-python库，但是在安装这个库的时候需要依赖setuptools库和six库，下面就要分别来下载这几个库

1、下载setuptools

打开这个网址会弹出类似下面的额下载窗口，选择保存文件，点击确定可以下载到setuptools-0.6c11-py2.6.egg

2、下载kafka-python

打开http://pipy.python.org,在搜索框里面输入kafka-python,然后点击【search】就打开如下图所示的界面。里面列出了对python版本的要求，但是根据测试，这个版本在Python 2.6.6下面也是可以正常运行的。

点击Download打开下面的界面

选择 kafka-python-1.3.5.tar.gz (md5) 开始下载

3、下载six

打开http://pipy.python.org,在搜索框里面输入six,然后点击【search】就打开如下图所示的界面。

打开six1.11.0

点击红色方框的链接，会下载到six-1.11.0.tar.gz

2、安装相关python库

在上一步里面我们已经下载了好相关的包，下面开始具体安装，首先创建一个/opt/package/python_lib，然后把这几个包文件上传到这里

1、安装setuptools

执行sh setuptools-0.6c11-py2.6.egg

执行结果如下：

setuptools安装成功。

2、安装six

1）解压

执行tar -zxvf six-1.11.0.tar.gz

解压之后会产生six-1.11.0文件夹

2）安装

cd six-1.11.0

ll

然后执行python setup.py install

3、安装kafka-python

执行tar -zxvf kafka-python-1.3.4.tar.gz解压安装包，会产生kafka-python-1.3.4文件夹，进入到该文件夹

执行python setup.py install

[root@node2 kafka-python-1.3.4]# python setup.py install
running install
running bdist_egg
running egg_info
creating kafka_python.egg-info
writing kafka_python.egg-info/PKG-INFO
writing top-level names to kafka_python.egg-info/top_level.txt
writing dependency_links to kafka_python.egg-info/dependency_links.txt
writing manifest file 'kafka_python.egg-info/SOURCES.txt'
reading manifest file 'kafka_python.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'kafka_python.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib
creating build/lib/kafka
copying kafka/future.py -> build/lib/kafka
copying kafka/client_async.py -> build/lib/kafka
copying kafka/errors.py -> build/lib/kafka
copying kafka/__init__.py -> build/lib/kafka
copying kafka/structs.py -> build/lib/kafka
copying kafka/context.py -> build/lib/kafka
copying kafka/cluster.py -> build/lib/kafka
copying kafka/conn.py -> build/lib/kafka
copying kafka/version.py -> build/lib/kafka
copying kafka/client.py -> build/lib/kafka
copying kafka/codec.py -> build/lib/kafka
copying kafka/util.py -> build/lib/kafka
copying kafka/common.py -> build/lib/kafka
creating build/lib/kafka/serializer
copying kafka/serializer/__init__.py -> build/lib/kafka/serializer
copying kafka/serializer/abstract.py -> build/lib/kafka/serializer
creating build/lib/kafka/partitioner
copying kafka/partitioner/hashed.py -> build/lib/kafka/partitioner
copying kafka/partitioner/roundrobin.py -> build/lib/kafka/partitioner
copying kafka/partitioner/__init__.py -> build/lib/kafka/partitioner
copying kafka/partitioner/base.py -> build/lib/kafka/partitioner
copying kafka/partitioner/default.py -> build/lib/kafka/partitioner
creating build/lib/kafka/consumer
copying kafka/consumer/__init__.py -> build/lib/kafka/consumer
copying kafka/consumer/base.py -> build/lib/kafka/consumer
copying kafka/consumer/group.py -> build/lib/kafka/consumer
copying kafka/consumer/simple.py -> build/lib/kafka/consumer
copying kafka/consumer/subscription_state.py -> build/lib/kafka/consumer
copying kafka/consumer/fetcher.py -> build/lib/kafka/consumer
copying kafka/consumer/multiprocess.py -> build/lib/kafka/consumer
creating build/lib/kafka/producer
copying kafka/producer/future.py -> build/lib/kafka/producer
copying kafka/producer/__init__.py -> build/lib/kafka/producer
copying kafka/producer/buffer.py -> build/lib/kafka/producer
copying kafka/producer/base.py -> build/lib/kafka/producer
copying kafka/producer/record_accumulator.py -> build/lib/kafka/producer
copying kafka/producer/simple.py -> build/lib/kafka/producer
copying kafka/producer/kafka.py -> build/lib/kafka/producer
copying kafka/producer/sender.py -> build/lib/kafka/producer
copying kafka/producer/keyed.py -> build/lib/kafka/producer
creating build/lib/kafka/vendor
copying kafka/vendor/socketpair.py -> build/lib/kafka/vendor
copying kafka/vendor/__init__.py -> build/lib/kafka/vendor
copying kafka/vendor/six.py -> build/lib/kafka/vendor
copying kafka/vendor/selectors34.py -> build/lib/kafka/vendor
creating build/lib/kafka/protocol
copying kafka/protocol/legacy.py -> build/lib/kafka/protocol
copying kafka/protocol/pickle.py -> build/lib/kafka/protocol
copying kafka/protocol/admin.py -> build/lib/kafka/protocol
copying kafka/protocol/struct.py -> build/lib/kafka/protocol
copying kafka/protocol/message.py -> build/lib/kafka/protocol
copying kafka/protocol/__init__.py -> build/lib/kafka/protocol
copying kafka/protocol/offset.py -> build/lib/kafka/protocol
copying kafka/protocol/metadata.py -> build/lib/kafka/protocol
copying kafka/protocol/fetch.py -> build/lib/kafka/protocol
copying kafka/protocol/commit.py -> build/lib/kafka/protocol
copying kafka/protocol/group.py -> build/lib/kafka/protocol
copying kafka/protocol/abstract.py -> build/lib/kafka/protocol
copying kafka/protocol/produce.py -> build/lib/kafka/protocol
copying kafka/protocol/api.py -> build/lib/kafka/protocol
copying kafka/protocol/types.py -> build/lib/kafka/protocol
creating build/lib/kafka/metrics
copying kafka/metrics/quota.py -> build/lib/kafka/metrics
copying kafka/metrics/kafka_metric.py -> build/lib/kafka/metrics
copying kafka/metrics/measurable.py -> build/lib/kafka/metrics
copying kafka/metrics/