Kafka入门，producer和consumer与hive

{

       "name":"hdfs-hive-sink-03",

       "config":{

              "connector.class":"io.confluent.connect.hdfs.HdfsSinkConnector",

              "tasks.max":"1",

              "topics":"text_source3",

              "hdfs.url":"hdfs://localhost:9000",

              "flush.size":"1",

              "hive.integration":"true",

              "hive.metastore.uris":"thrift://localhost:9083",

              "schema.compatibility":"BACKWARD"

             

       }

}

 

 

kafka-avro-console-producer --broker-list localhost:9092 --topic text_source3 --property vae.schema='{"type":"record","name":"myrecord","fields":[{"name":"f1","type":"string"}]}'

 

{"f1":"f1_001"}

 

输入（生产者）

输出（消费者）

 

 

 

 

 

 

 

kafka-topics --delete --zookeeper localhost:2181 --topic test_source

删除topic

kafka-topics --zookeeper localhost:2181 --list

查询topic