Kafka系列之:Kafka Connect Configs
- 一、Kafka Connect配置详细总结
- 1.config.storage.topic
- 2.group.id
- 3.key.converter
- 4.offset.storage.topic
- 5.status.storage.topic
- 6.value.converter
- 7.bootstrap.servers
- 8.heartbeat.interval.ms
- 9.rebalance.timeout.ms
- 10.session.timeout.ms
- 11.ssl.key.password
- 12.ssl.keystore.certificate.chain
- 13.ssl.keystore.key
- 14.ssl.keystore.location
- 15.ssl.keystore.password
- 16.ssl.truststore.certificates
- 17.ssl.truststore.location
- 18.ssl.truststore.password
- 19.connections.max.idle.ms
- 20.connector.client.config.override.policy
- 21.receive.buffer.bytes
- 22.request.timeout.ms
- 23.sasl.client.callback.handler.class
- 24.sasl.jaas.config
- 25.sasl.kerberos.service.name
- 26.sasl.login.class
- 27.sasl.mechanism
- 28.security.protocol
- 29.send.buffer.bytes
- 30.ssl.enabled.protocols
- 31.worker.sync.timeout.ms
- 32.worker.unsync.backoff.ms
- 33.access.control.allow.methods
- 34.access.control.allow.origin
- 35.admin.listeners
- 36.client.id
- 37.config.providers
- 38.config.storage.replication.factor
- 39.connect.protocol
- 40.header.converter
- 41.listeners
- 42.metadata.max.age.ms
- 43.metric.reporters
- 44.metrics.num.samples
- 45.metrics.recording.level
- 46.metrics.sample.window.ms
- 47.offset.flush.interval.ms
- 48.offset.flush.timeout.ms
- 49.offset.storage.partitions
- 50.offset.storage.replication.factor
- 51.plugin.path
- 52.reconnect.backoff.max.ms
- 53.reconnect.backoff.ms
- 54.response.http.headers.config
- 55.rest.advertised.host.name
- 56.rest.advertised.listener
- 57.rest.advertised.port
- 58.retry.backoff.ms
- 59.sasl.kerberos.kinit.cmd
- 60.sasl.kerberos.min.time.before.relogin
- 61.scheduled.rebalance.max.delay.ms
- 62.status.storage.replication.factor
- 63.topic.creation.enable
- 64.topic.tracking.enable
- 二、Source Connect配置详细总结
- 1.name
- 2.connector.class
- 3.tasks.max
- 4.key.converter
- 5.value.converter
- 6.header.converter
- 7.config.action.reload
- 8.transforms
- 9.predicates
- 10.errors.retry.timeout
- 11.errors.retry.delay.max.ms
- 12.errors.tolerance
- 13.errors.log.enable
- 14.errors.log.include.messages
- 15.topic.creation.groups
- 三、Sink Connector配置
- 1.name
- 2.connector.class
- 3.tasks.max
- 4.topics
- 5.topics.regex
- 6.key.converter
- 7.value.converter
- 8.header.converter
- 9.config.action.reload
- 10.transforms
- 11.predicates
- 12.errors.retry.timeout
- 13.errors.retry.delay.max.ms
- 14.errors.tolerance
- 15.errors.log.enable
- 16.errors.log.include.messages
- 17.errors.deadletterqueue.topic.name
- 18.errors.deadletterqueue.topic.replication.factor
- 19.errors.deadletterqueue.context.headers.enable
下面是 Kafka Connect 框架的配置。
一、Kafka Connect配置详细总结
1.config.storage.topic
- 存储连接器配置的 Kafka 主题的名称
- Type: string
- Importance: high
2.group.id
- 一个唯一的字符串,用于标识此工作人员所属的 Connect 集群组。
- Type: string
- Importance: high
3.key.converter
- 转换器类用于在 Kafka Connect 格式和写入 Kafka 的序列化格式之间进行转换。这控制了写入 Kafka 或从 Kafka 读取的消息中键的格式,并且由于它独立于连接器,它允许任何连接器使用任何序列化格式。常见格式的示例包括 JSON 和 Avro。
- Type: class
- Importance: high
4.offset.storage.topic
- 存储连接器偏移量的 Kafka 主题的名称
- Type: string
- Importance: high
5.status.storage.topic
- 存储连接器和任务状态的 Kafka 主题的名称
- Type: string
- Importance: high