kafka 新手入门

 注意：kafka版本0.10.2.0
下载地址:http://kafka.apache.org/downloads，https://www.apache.org/dyn/closer.cgi?path=/kafka/0.10.2.0/kafka_2.11-0.10.2.0.tgz
 tar -xzf kafka_2.11-0.10.2.0.tgz
 cd kafka_2.11-0.10.2.0
 cd config
 vim server.properties
 修改参数#advertised.listeners=PLAINTEXT://your.host.name:9092 打开注释写上服务器IP
 启动zookeeper bin/zookeeper-server-start.sh config/zookeeper.properties
 启动kafka bin/kafka-server-start.sh config/server.properties
 shell测试创建主题
 ./kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic helloKafka
 物理位置在server.properties的log.dirs配置 默认路径log.dirs=/tmp/kafka-logs
 cd /tmp/kafka-logs目录下可以看见helloKafka-0文件夹，--replication-factor 1副本数量单机环境略过 --partitions 1 表示分区数(--partitions 2 对应创建文件 helloKafka-0  helloKafka-1)
 查看创建的topic ./kafka-topics.sh --list --zookeeper localhost:2181
 shell测试Producer
 bin/kafka-console-producer.sh --broker-list localhost:9092 --topic helloKafka
 hello kafka
 shell启动消费者
 bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic helloKafka --from-beginning
 hello kafka
 也可在helloKafka-0文件夹下查看.log结尾的文件保存发送的信息
 
 java发送消息到kafka
 引入生产者jar包
 <dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-clients</artifactId>
<version>0.10.2.0</version>
 </dependency>
  Producer链接配置文件kafkaProducer.properties
  bootstrap.servers=your.host.name:9092 //kafka服务器地址
  acks=all
  retries=1
  batch.size=163840
  linger.ms=2
  buffer.memory=335544320
  metadata.fetch.timeout.ms=3000
  key.serializer=org.apache.kafka.common.serialization.StringSerializer
  value.serializer=org.apache.kafka.common.serialization.StringSerializer
 
  Java生产者代码
  public static void main( String[] args )
    {
        
    Properties props = new Properties();
    InputStream in  = null;
        try {
            //加载发送者的配置列表
        in = new FileInputStream(new File("kafkaProducer.properties"));
props.load(in);
} catch (IOException e) {

e.printStackTrace();
}finally {
try {
in.close();
} catch (IOException e) {
e.printStackTrace();
}finally {
if (in != null) {
in = null;
}

}
} 
   
   
        
KafkaProducer<String, String> producer = new KafkaProducer<String, String>(props);
   
  
producer.send(new ProducerRecord<String, String>("helloKafka", "i like coco"), new Callback(){


public void onCompletion(RecordMetadata metadata, Exception exception) {
if(exception != null){

System.out.println("call back !!!!!!!!@@@@@@@@@@@@@@");




}else{


System.out.println("------------------------生产者开始---------------------------");
System.out.println("The offset of the record we just sent is: " + metadata.offset());
System.out.println("The topic of the record we just sent is: " + metadata.topic());
System.out.println("The partition of the record we just sent is: " + metadata.partition());
System.out.println("The hashCode of the record we just sent is: " + metadata.hashCode());
System.out.println("------------------------生产者结束---------------------------");
}

}

});
   
    }