apache kafka安装

apache flink网站上的例子需要安装kafka, 按照https://kafka.apache.org/quickstart 安装启动server报内存不足

bin/zookeeper-server-start.sh config/zookeeper.properties 

bin/kafka-server-start.sh config/server.properties 

然后进入kafka_2.12-0.11.0.0的bin目录,找到kafka-server-start.sh文件,vi kafka-server-start.sh

找到这一行export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G" 

改为   export KAFKA_HEAP_OPTS="-Xmx256M -Xms128M"  正常启动

1、创建主题

bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

2.查看主题

bin/kafka-topics.sh --list --zookeeper localhost:2181 

 3.生产者生产消息

 bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test

4.消费者取出消息 

bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning 

 

Set up Apache Kafka clusters and develop custom message producers and consumers using practical, hands-on examples Overview Write custom producers and consumers with message partition techniques Integrate Kafka with Apache Hadoop and Storm for use cases such as processing streaming data Provide an overview of Kafka tools and other contributions that work with Kafka in areas such as logging, packaging, and so on In Detail Message publishing is a mechanism of connecting heterogeneous applications together with messages that are routed between them, for example by using a message broker like Apache Kafka. Such solutions deal with real-time volumes of information and route it to multiple consumers without letting information producers know who the final consumers are. Apache Kafka is a practical, hands-on guide providing you with a series of step-by-step practical implementations, which will help you take advantage of the real power behind Kafka, and give you a strong grounding for using it in your publisher-subscriber based architectures. Apache Kafka takes you through a number of clear, practical implementations that will help you to take advantage of the power of Apache Kafka, quickly and painlessly. You will learn everything you need to know for setting up Kafka clusters. This book explains how Kafka basic blocks like producers, brokers, and consumers actually work and fit together. You will then explore additional settings and configuration changes to achieve ever more complex goals. Finally you will learn how Kafka works with other tools like Hadoop, Storm, and so on. You will learn everything you need to know to work with Apache Kafka in the right format, as well as how to leverage its power of handling hundreds of megabytes of messages per second from multiple clients. What you will learn from this book Download and build Kafka Set up single as well as multi-node Kafka clusters and send messages Learn Kafka design internals and message compression Understand how replication works in Kafka Write Kafka message producers and consumers using the Kafka producer API Get an overview of consumer configurations Integrate Kafka with Apache Hadoop and Storm Use Kafka administration tools Approach The book will follow a step-by-step tutorial approach which will show the readers how to use Apache Kafka for messaging from scratch. Who this book is written for Apache Kafka is for readers with software development experience, but no prior exposure to Apache Kafka or similar technologies is assumed. This book is also for enterprise application developers and big data enthusiasts who have worked with other publisher-subscriber based systems and now want to explore Apache Kafka as a futuristic scalable solution. Product Details Paperback: 88 pages Publisher: Packt Publishing (October 17, 2013) Language: English ISBN-10: 1782167935 ISBN-13: 978-1782167938 Product Dimensions: 9.2 x 7.5 x 0.2 inches
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值