kafka java 对象_Kafka序列化一个对象

我开始和 Kafka 一起玩 . 我已经设置了一个zookeeper配置，我设法发送和使用String消息 . 现在我试图传递一个Object(在java中)，但由于某种原因，在消费者中解析Message时我有头问题 . 我尝试了几个序列化选项(使用Decoder / Encoder)，并且所有返回相同的头问题 .

这是我的代码 生产环境 者：

Properties props = new Properties();

props.put("zk.connect", "localhost:2181");

props.put("serializer.class", "com.inneractive.reporter.kafka.EventsDataSerializer");

ProducerConfig config = new ProducerConfig(props);

Producer producer = new Producer(config);

ProducerData data = new ProducerData("test3", 1, Arrays.asList(new EventDetails());

try {

producer.send(data);

} finally {

producer.close();

}

和消费者：

Properties props = new Properties();

props.put("zk.connect", "localhost:2181");

props.put("zk.connectiontimeout.ms", "1000000");

props.put("groupid", "test_group");

// Create the connection to the cluster

ConsumerConfig consumerConfig = new ConsumerConfig(props);

ConsumerConnector consumerConnector = Consumer.createJavaConsumerConnector(consumerConfig);

// create 4 partitions of the stream for topic “test”, to allow 4 threads to consume

Map>> topicMessageStreams =

consumerConnector.createMessageStreams(ImmutableMap.of("test3", 4), new EventsDataSerializer());

List> streams = topicMessageStreams.get("test3");

// create list of 4 threads to consume from each of the partitions

ExecutorService executor = Executors.newFixedThreadPool(4);

// consume the messages in the threads

for (final KafkaMessageStream stream: streams) {

executor.submit(new Runnable() {

public void run() {

for(EventDetails event: stream) {

System.err.println("********** Got message" + event.toString());

}

}

});

}

和我的序列化器：

public class EventsDataSerializer implements Encoder, Decoder {

public Message toMessage(EventDetails eventDetails) {

try {

ObjectMapper mapper = new ObjectMapper(new SmileFactory());

byte[] serialized = mapper.writeValueAsBytes(eventDetails);

return new Message(serialized);

} catch (IOException e) {

e.printStackTrace();

return null; // TODO

}

}

public EventDetails toEvent(Message message) {

EventDetails event = new EventDetails();

ObjectMapper mapper = new ObjectMapper(new SmileFactory());

try {

//TODO handle error

return mapper.readValue(message.payload().array(), EventDetails.class);

} catch (IOException e) {

e.printStackTrace();

return null;

}

}

}

这是我得到的错误：

org.codehaus.jackson.JsonParseException: Input does not start with Smile format header (first byte = 0x0) and parser has REQUIRE_HEADER enabled: can not parse

at [Source: N/A; line: -1, column: -1]

当我使用 MessagePack 并且简单地写入 ObjectOutputStream 时，我遇到了类似的 Headers 问题 . 我还尝试将有效负载CRC32添加到消息中，但这也无济于事 .

我在这做错了什么？