Spark 连接kafka报错: Error while fetching metadata with correlation id xx [topicName=INVALID_REPLICATIO]

问题来源

同事在一次spark程序中，通过spark在kafka中的topic中写入数据发生报错，错误日志如下:

2021-01-25 19:58:11,794 INFO utils.AppInfoParser: Kafka version : 2.0.0-cdh6.1.1
2021-01-25 19:58:11,794 INFO utils.AppInfoParser: Kafka commitId : null
2021-01-25 19:58:11,984 INFO codegen.CodeGenerator: Code generated in 80.049669 ms
2021-01-25 19:58:12,020 WARN clients.NetworkClient: [Producer clientId=producer-1] Connection to node -3 could not be established. Broker may not be available.
2021-01-25 19:58:12,034 WARN clients.NetworkClient: [Producer clientId=producer-1] Error while fetching metadata with correlation id 4 : {tagging-api-1611575841019=INVALID_REPLICATION_FACTOR}
2021-01-25 19:58:12,034 INFO clients.Metadata: Cluster ID: aKKlHlDqQtalfjbLYRW1GQ
2021-01-25 19:58:12,136 WARN clients.NetworkClient: [Producer clientId=producer-1] Error while fetching metadata with correlation id 9 : {tagging-a