Kafka在电信BOSS计费系统的应用

最新推荐文章于 2024-01-03 13:44:40 发布

倪某某

最新推荐文章于 2024-01-03 13:44:40 发布

阅读量308

点赞数

分类专栏： BSS 电信计费

本文链接：https://blog.csdn.net/ethanni/article/details/111164096

版权

本文介绍了Kafka在电信BOSS计费系统的应用，阐述了Kafka作为事件流处理平台的功能，包括数据集成和流处理。Kafka提供了高吞吐、高性能和高可用性，但其依赖Zookeeper、运维复杂和可能丢数据的缺点也需要注意。在电信领域，Kafka常用于大数据实时计算和日志采集，与Hive和Hbase协作处理海量数据。同时，文章提到了Rocket MQ作为在高可靠性场景下的替代选择。

摘要由CSDN通过智能技术生成

Kafka的背景：

kafka的诞生，是为了解决linkedin的数据管道问题，起初linkedin的技术架构师jay creps采用了ActiveMQ来进行数据交换，大约是在2010年前后，那时的ActiveMQ还远远无法满足linkedin对数据传递系统的要求，经常由于各种缺陷而导致消息阻塞或者服务无法正常访问，为了能够解决这个问题，linkedin决定研发自己的消息传递系统，当时linkedin的首席架构师jay kreps便开始组织团队进行消息传递系统的研发；

一开始的目的是日志收集和传输（1.互联网页面page view等点击，浏览信息 2.服务器CPU，内存等类似于网管信息）

kafka是什么:

Apache Kafka® is an event streaming platform,capturing data in real-time from event sources like databases, sensors, mobile devices, cloud services, and software applications in the form of streams of events; storing these event streams durably for later retrieval; manipulating, processing, and reacting to the event streams in real-time as well as retrospectively; and routing the event streams to different destination technologies as needed.

Kafka是事件流实时处理平台，支持数据的实时获取，存储，处理。

Kafka combines three key capabilities so you can implement your use cases for event streaming end-to-end with a single battle-tested solution:

To publish (write) and subscribe to (read) streams of events, including continuous import/export of your data from other systems.
To store streams of events durably and reliably for as long as you want.
To process streams of events as they occur or retrospectively

Produc