Step 1: Download the code
Download the 1.1.0 release and un-tar it.
1 2 | > tar -xzf kafka_2.11-1.1.0.tgz > cd kafka_2.11-1.1.0 |
Step 2: Start the server
> bin/zookeeper-server-start.sh config/zookeeper.properties [2013-04-22 15:01:37,495] INFO Reading configuration from: config/zookeeper.properties (org.apache.zookeeper.server.quorum.QuorumPeerConfig) ... |
Now start the Kafka server:
1 2 3 4 | > bin/kafka-server-start.sh config/server.properties [2013-04-22 15:01:47,028] INFO Verifying properties (kafka.utils.VerifiableProperties) [2013-04-22 15:01:47,051] INFO Property socket.send.buffer.bytes is overridden to 1048576 (kafka.utils.VerifiablePr |
Step 3: Create a topic
Let's create a topic named "test" with a single partition and only one replica:
1 | > bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test |
We can now see that topic if we run the list topic command:
1 2 | > bin/kafka-topics.sh --list --zookeeper localhost:2181 test |
Step 4: Send some messages
Run the producer and then type a few messages into the console to send to the server.
1 2 3 | > bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test This is a message This is another message |
Step 5: Start a consumer
Kafka also has a command line consumer that will dump out messages to standard output.
1 2 3 | > bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning This is a message This is another message |
Step 6: Use Kafka Connect to import/export data
Kafka Connect is a tool included with Kafka that imports and exports data to Kafka
> bin/connect-standalone.sh config/connect-standalone.properties config/connect-file-source.properties config/connect-file-sink.properties |
第一个是Kafka Connect过程的配置,包含常见的配置,如Kafka代理连接到数据的序列化格式。
其余的配置文件都指定了要创建的连接器。
这些文件包括一个惟一的连接器名称、实例化的连接器类以及连接器需要的任何其他配置。
Step 7: Use Kafka Streams to process data
Kafka Streams是用于构建关键任务实时应用程序和微服务的客户端库,其中的输入和/或输出数据存储在Kafka集群中。Kafka Streams结合了在客户端编写和部署标准Java和Scala应用程序的简单性,以及Kafka的服务器端集群技术的优点,使这些应用程序具有高度可伸缩性、弹性、容错、分布式和更多的优点。这个快速启动示例将演示如何在这个库中运行流应用程序