1.kafka connector基本架构模型图
topic log
consumer
connector
Kafka connect有两个概念,一个source,另一个是sink。source是把数据从一个系统拷贝到kafka里,sink是从kafka拷贝到另一个系统里
Connectors – the high level abstraction that coordinates data streaming by managing tasks
Tasks – the implementation of how data is copied to or from Kafka
Workers – the running processes that execute connectors and tasks
Converters – the code used to translate data between Connect and the system sending or receiving data
Converters
2.kafka consumer与connector区别
Kafka Connect provides built-in support for parallelism and scalable data copying
每个task在独立线程中启动consumer,不断轮询;
3.connector sink流程图