Debezium is a distributed platform that turns your existing databases into event streams, so applications can see and respond immediately to each row-level change in the databases.
Debezium is built on top of Apache Kafka and provides Kafka Connect compatible connectors that monitor specific database management systems. Debezium records the history of data changes in Kafka logs, from where your application consumes them. This makes it possible for your application to easily consume all of the events correctly and completely. Even if your application stops unexpectedly, it will not miss anything: when the application restarts, it will resume consuming the events where it left off.
Debezium是什么?
Debezium是一个分布式平台,它将您现有的数据库转换为event流,因此应用程序可以看到数据库中的每一个行级更改并立即做出响应。Debezium构建在Apache Kafka之上,并提供Kafka连接兼容的连接器来jian视特定的数据库管理系统。Debezium在Kafka日志中记录数据更改的历史,您的应用程序将从这里使用它们。这使您的应用程序能够轻松、正确、完整地使用所有event。即使您的应用程序停止(或beng溃),在重新启动时,它将开始消耗它停止的event,因此它不会错过任何东西。
Debezium是一个开源项目,为bu获数据更改(Capture Data Change,CDC)提供了一个低延迟的流式处理平台,通过安装配置Debezium监控数据库,可以实时消费行级别(row-level)的更改。身为一个分布式系统,Debezium也拥有良好的容错性。
Debezium的源端(即支持jian控哪些数据库) : MySQL,MongoDB,PostgreSQL,Oracle,SQL Server
Debezium的目标端(即可以数据导入端) : Kafka
What are some uses of Debezium?
The primary use of Debezium is to enable applications to respond almost immediately whenever data
in databases change. Applications can do anything with the insert, update, and delete events.
They might use the events to know when to remove entries from a cache. They might update search indexes
with the data. They might update a derived data store with the same information or with information
computed from the changing data, such as with Command Query Responsibility Separation (CQRS).
They might send a push notification to one or more mobile devices. They might aggregate the changes
and produce a stream of patches for entities.
适用场景:
可以帮助我们解决好多数据实时变更处理、数据分析、微服务的数据通信
缓存独立(Cache invalidation)
简化单体应用(Simplifying monolithic applications)
数据库切片(Sharing databases)
数据集成(Data integration)
命令查询职责分离(Command Query Responsibility Separation)
Debezium ,可以把来自 MySQL、PostgreSQL、Oracle、Microsoft SQL Server 和许多其他数据库的更改实时流式传输到 Kafka 中。 Debezium 为变更日志提供了统一的格式结构,并支持使用 JSON 和 Apache Avro 序列化消息。
Debezium是Apache Kafka Connect的一组源连接器,使用change data capture (CDC)从不同的数据库中获取更改。与其他方法如轮询或双写不同,基于日志的CDC由Debezium实现:
确保捕获所有数据更改
以非常低的延迟(例如,MySQL或Postgres的ms范围)生成更改event,同时避免增加频繁轮询的CPU使用量
不需要更改数据模型(如“最后更新”列)
可以bu获删除
可以bu获旧记录状态和其他元数据,如事务id和引发查询(取决于数据库的功能和配置)
4767

被折叠的 条评论
为什么被折叠?



