学习kafka：log4j写入kafka

最近花了一天时间把kafka官方文档看了一遍，自己测试了一下，特记录与此，原理概念看完文档就能理解，主要是问题。
1 版本POM配置 ，注意jar包可能会冲突：

 <dependency>
            <groupId>org.apache.kafka</groupId>
            <artifactId>kafka_2.10</artifactId>
            <version>0.8.2.2</version>
            <!--<exclusions>-->
            <!--<exclusion>-->
            <!--<groupId>org.slf4j</groupId>-->
            <!--<artifactId>slf4j-log4j12</artifactId>-->
            <!--</exclusion>-->
            <!--</exclusions>-->
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.apache.kafka/kafka-log4j-appender -->
        <dependency>
            <groupId>org.apache.kafka</groupId>
            <artifactId>kafka-log4j-appender</artifactId>
            <version>0.10.2.1</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-log4j12</artifactId>
            <version>1.7.5</version>
        </dependency>

2 log4j配置：

log4j.rootLogger=INFO,console,KAFKA
## appender KAFKA
log4j.appender.KAFKA=kafka.producer.KafkaLog4jAppender
log4j.appender.KAFKA.topic=test
log4j.appender.KAFKA.brokerList=127.0.0.1:9092
log4j.appender.KAFKA.pa=127.0.0.1:9092

log4j.appender.KAFKA.compressionType=none
log4j.appender.KAFKA.syncSend=true
log4j.appender.KAFKA.layout=org.apache.log4j.PatternLayout
log4j.appender.KAFKA.ThresholdFilter.level=INFO
log4j.appender.KAFKA.ThresholdFilter.onMatch=ACCEPT
log4j.appender.KAFKA.ThresholdFilter.onMismatch=DENY

log4j.appender.KAFKA.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L %% - %m%n

## appender console
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.target=System.out
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%d (%t) [%p - %l] %m%n

3 测试代码：

package com.need.kafka.services.demo;

import com.need.kafka.services.utils.KafkaPropertiesManager;
import kafka.javaapi.producer.Producer;
import kafka.producer.KeyedMessage;
import kafka.producer.ProducerConfig;
import org.slf4j.impl.StaticLoggerBinder;
import org.slf4j.*;

import java.util.ArrayList;
import java.util.List;

/**
 * 这个是demo 主要用来测试生产者的单条信息发送和批量信息发送
 * <p/>
 */
public class ProducerDemo {

    private static final Logger log = LoggerFactory.getLogger(ProducerDemo.class);

    public static void main(String args[]) throws InterruptedException {
//        ProducerConfig producerConfig = new ProducerConfig(KafkaPropertiesManager.getProducerProps());
//        Producer<String, String> producer = new Producer<String, String>(producerConfig);
//        // 单个发送
//        for (int i = 0; i <= 10; i++) {
//            KeyedMessage<String, String> message =
//                    new KeyedMessage<String, String>(KafkaProperties.TEST_TOPIC, i + "", "Message" + i);
//            producer.send(message);
//        }
//
        // 批量发送
//        List<KeyedMessage<String, String>> messages = new ArrayList<KeyedMessage<String, String>>(100);
//        for (int i = 0; i <= 100; i++) {
//            KeyedMessage<String, String> message =
//                    new KeyedMessage<String, String>(KafkaProperties.TEST_TOPIC, i + "", "Message" + i);
//            messages.add(message);
//            if (i % 100 == 0) {
//                producer.send(messages);
//                messages.clear();
//            }
//        }
//        producer.send(messages);
        for(int i = 0;i <= 10; i++) {
                log.info("trace:This is Message [" + i + "] from log4j producer .. ");
                log.info("This is Message [" + i + "] from log4j producer .. ");
        }
    }
}

注意：类的来源，有的类不同jar都会有

4结果：

2017-05-21 10:41:09 INFO  ProducerDemo:44 % - trace:This is Message [5] from log4j producer ..

2017-05-21 10:41:09 INFO  ProducerDemo:45 % - This is Message [5] from log4j producer ..

2017-05-21 10:41:09 INFO  ProducerDemo:44 % - trace:This is Message [6] from log4j producer ..

2017-05-21 10:41:09 INFO  ProducerDemo:45 % - This is Message [6] from log4j producer ..

2017-05-21 10:41:09 INFO  ProducerDemo:44 % - trace:This is Message [7] from log4j producer ..

2017-05-21 10:41:09 INFO  ProducerDemo:45 % - This is Message [7] from log4j producer ..

2017-05-21 10:41:09 INFO  ProducerDemo:44 % - trace:This is Message [8] from log4j producer ..

2017-05-21 10:41:09 INFO  ProducerDemo:45 % - This is Message [8] from log4j producer ..

2017-05-21 10:41:09 INFO  ProducerDemo:44 % - trace:This is Message [9] from log4j producer ..

2017-05-21 10:41:09 INFO  ProducerDemo:45 % - This is Message [9] from log4j producer ..

2017-05-21 10:41:09 INFO  ProducerDemo:44 % - trace:This is Message [10] from log4j producer ..

2017-05-21 10:41:09 INFO  ProducerDemo:45 % - This is Message [10] from log4j producer ..

注释：最麻烦的可能就是原项目已经使用了log4j，引入了一堆jar，可能需要重新修改jar的引入。