You’d better go to the receptionist desk on the first floor to check in.
That company puts all new employees on a three-month probation.
Junior college degree Bachelor degree
Apache Kafka是一个开源的分布式事件流平台,被数千家公司用于高性能数据管道、流分析、数据集成和任务关键型应用程序Apache Kafka is an open-source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications
Technically speaking, event streaming is the practice of capturing data in real-time from event sources like databases, sensors, mobile devices, cloud services, and software applications in the form of streams of events; storing these event streams durably for later retrieval从技术上讲,事件流是以事件流的形式从数据库、传感器、移动设备、云服务和软件应用程序等事件源实时捕获数据的实践;持久地存储这些事件流以供以后检索
real-time 实时的,实时的
retrospectively 回溯的
To process payments and financial transactions in real-time, such as in stock exchanges, banks, and insurances.实时处理付款和金融交易,如证券交易所、银行和保险。
To track and monitor cars, trucks, fleets, and shipments in real-time, such as in logistics and the automotive industry.实时跟踪和监控汽车、卡车、车队和货运,如物流和汽车行业。
To continuously capture and analyze sensor data from IoT devices or other equipment, such as in factories and wind parks.
从物联网设备或其他设备(如工厂和风电场)连续捕获和分析传感器数据。
To collect and immediately react to customer interactions and orders, such as in retail, the hotel and travel industry, and mobile applications.收集并立即响应客户的互动和订单,如零售、酒店和旅游业以及移动应用程序。
To monitor patients in hospital care and predict changes in condition to ensure timely treatment in emergencies.监测医院护理中的患者并预测病情变化,以确保在紧急情况下及时治疗。
divisions 分部
To connect, store, and make available data produced by different divisions of a company.连接、存储和提供公司不同部门产生的数据。
To serve as the foundation for data platforms, event-driven architectures, and microservices.
作为数据平台、事件驱动架构和微服务的基础。
a single battle-tested solution经过实战检验的解决方案
To publish (write) and subscribe to (read) streams of events, including continuous import/export of your data from other systems.发布(写入)和订阅(读取)事件流,包括从其他系统连续导入/导出数据。
To store streams of events durably and reliably for as long as you want.根据需要持久可靠地存储事件流。
To process streams of events as they occur or retrospectively.在事件发生或回顾时处理事件流。
And all this functionality is provided in a distributed, highly scalable, elastic, fault-tolerant, and secure manner所有这些功能都是以分布式、高度可扩展、弹性、容错和安全的方式提供的
Kafka can be deployed on bare-metal hardware, virtual machines, and containers, and on-premises as well as in the cloud. You can choose between self-managing your Kafka environments and using fully managed services offered by a variety of vendors.Kafka可以部署在裸机硬件、虚拟机和容器上,也可以部署在本地和云中。您可以在自我管理Kafka环境和使用各种供应商提供的完全管理服务之间进行选择。
Terminology ˌtɜːmɪˈnɒlədʒi 专业术语
Main Concepts and Terminology主要概念和术语
Producers are those client applications that publish (write) events to Kafka, and consumers are those that subscribe to (read and process) these events. In Kafka, producers and consumers are fully decoupled and agnostic of each other, which is a key design element to achieve the high scalability that Kafka is known for生产者是那些向Kafka发布(写入)事件的客户端应用程序,消费者是那些订阅(读取和处理)这些事件的应用程序。在Kafka中,生产者和消费者完全解耦,互不可知,这是实现Kafka所熟知的高可伸缩性的关键设计元素
For example, producers never need to wait for consumers. Kafka provides various guarantees such as the ability to process events exactly-once.例如,生产者永远不需要等待消费者。Kafka提供了各种保证,例如能够准确处理一次事件。
perfectly ˈpɜːfɪktli 完全的
Kafka’s performance is effectively constant with respect to data size, so storing data for a long time is perfectly fine.Kafka的性能在数据大小方面实际上是恒定的,因此长时间存储数据是非常好的。
Topics are partitioned, meaning a topic is spread over a number of “buckets” located on different Kafka brokers. This distributed placement of your data is very important for scalability because it allows client applications to both read and write the data from/to many brokers at the same time. When a new event is published to a topic, it is actually appended to one of the topic’s partitions. Events with the same event key (e.g., a customer or vehicle ID) are written to the same partition, and Kafka guarantees that any consumer of a given topic-partition will always read that partition’s events in exactly the same order as they were written.主题是分区的,这意味着一个主题分布在位于不同Kafka代理上的多个“桶”上。数据的这种分布式放置对于可扩展性非常重要,因为它允许客户端应用程序同时从多个代理读取数据和向多个代理写入数据。当新事件发布到主题时,它实际上会附加到主题的一个分区。具有相同事件密钥(例如,客户或车辆ID)的事件被写入同一分区,Kafka保证给定主题分区的任何使用者都将始终以与写入时完全相同的顺序读取该分区的事件。
Figure: This example topic has four partitions P1–P4. Two different producer clients are publishing, independently from each other, new events to the topic by writing events over the network to the topic’s partitions. Events with the same key (denoted by their color in the figure) are written to the same partition. Note that both producers can write to the same partition if appropriate.图:本示例主题有四个分区P1–P4。两个不同的生产者客户端通过将事件通过网络写入主题的分区,彼此独立地向主题发布新事件。具有相同键的事件(在图中由其颜色表示)被写入同一分区。注意,如果合适的话,两个生产者都可以写入同一个分区。
To make your data fault-tolerant and highly-available, every topic can be replicated, even across geo-regions or datacenters, so that there are always multiple brokers that have a copy of the data just in case things go wrong, you want to do maintenance on the brokers, and so on. A common production setting is a replication factor of 3, i.e., there will always be three copies of your data. This replication is performed at the level of topic-partitions为了使您的数据具有容错性和高可用性,每个主题都可以复制,甚至可以跨地理区域或数据中心复制,以便始终有多个代理拥有数据的副本,以防出现问题、您希望对代理进行维护等。常见的生产设置是复制因子为3,即始终有三个数据副本。此复制在主题分区级别执行
This primer should be sufficient for an introduction. The Design section of the documentation explains Kafka’s various concepts in full detail, if you are interested. sufficient səˈfɪʃnt 足够的 concepts ˈkɑnsɛpts 概念
In addition to command line tooling for management and administration tasks, Kafka has five core APIs for Java and Scala:除了用于管理和管理任务的命令行工具外,Kafka还为Java和Scala提供了五个核心API:
reusable ˌriːˈjuːzəbl 可重复使用的
The Kafka Connect API to build and run reusable data import/export connectors that consume (read) or produce (write) streams of events from and to external systems and applications so they can integrate with Kafka. For example, a connector to a relational database like PostgreSQL might capture every change to a set of tables. However, in practice, you typically don’t need to implement your own connectors because the Kafka community already provides hundreds of ready-to-use connectors.Kafka Connect API用于构建和运行可重复使用的数据导入/导出连接器,这些连接器使用(读取)或生成(写入)来自外部系统和应用程序的事件流,以便与Kafka集成。例如,关系数据库(如PostgreSQL)的连接器可能会捕获对一组表的每个更改。然而,在实践中,您通常不需要实现自己的连接器,因为Kafka社区已经提供了数百个现成的连接器。
To get hands-on experience with Kafka, follow the Quickstart.要获得Kafka的亲身体验,请遵循Quickstart。
academic papers. 学术论文
To understand Kafka in more detail, read the Documentation. You also have your choice of Kafka books and academic papers.