kafka和mqtt的区别和联系

参考：https://www.zhihu.com/question/30343125

两者虽然都是从传统的Pub/Sub消息系统演化出来的，但是进化的方向不一样，以下是几个比较突出的点：

1. Kafka是为了数据集成的场景，与以往Pub/Sub消息总线不一样，通过分布式架构提供了海量消息处理、高容错的方式存储海量数据流、保证数据流的顺序等特性。可以参考云上的卡夫卡 - 数据工会。
2. MQTT是为了物联网场景而优化，不但提供多个QoS选项（exact once、at least once、at most once），而且还有层级主题、遗嘱等等特性。可以参考MQTT入门篇 - 数据工会。

说白了都是传统消息系统（老爸）的子嗣，只是与不同的场景（老妈）结合的产物。不过，两者却可以结合起来使用。比如可以用MQTT接受物联网设备上传的数据，然后接入Kafka，最后可以同时分发到HDFS归档、数据仓库做OLAP分析、Elasticsearch做全文检索，这样的架构非常适合大型物联网项目，不但能够处理海量数据同时也具有很好的扩展性。

MQTT and Kafka, on paper, are publish/subscribe (pub/sub) messaging platforms. But a key distinction between the two relates to how they are put to use. Unlike Kafka, MQTT is a machine-to-machine (M2M) messaging protocol. This means that while Kafka is responsible for data management and ingestion into computation systems, MQTT specializes in data packet exchange between two physical devices. In our home automation example, if we want to adjust air conditioning according to ambient temperature, the temperature sensor must be able to communicate with the air conditioner. This communication is held through MQTT. As expected, MQTT is only intended for small-sized data sets, where responsiveness and power efficiency are prioritized. This doesn’t lend well to large data sets – the result being bad scaling. Kafka, on the other hand, is designed specifically with massive data feeds in mind. Sound IoT systems use Kafka and MQTT in conjunction, co-existing separately for maximum performance and scalability.

转载于:https://www.cnblogs.com/ryu-manager/p/9481201.html