Uber 有超大量的实时数据需要分析,(路况分析,计算车辆到达需要的时间等)
有More than one trillion real-time 消息通过他们的kafka,so 他们需要一个infrastructure 并且infrastructure (平台)需要有如下特性:
(1) easily navigable(通航) by all users regardless(而不用管) of technical expertise(专门知识)
(2) scalable and efficient enough to analyze real-time events, and (3) robust enough to continuously support hundreds if not thousands of critical(危机) jobs.
athenaX运行流程step below
AthenaX’s workflow follows the steps below:
- Users specify a job in SQL and submit it to the AthenaX master.
- The AthenaX master validates the query and compiles it down to a Flink job.
- The AthenaX master packages, deploys, and executes the job in the YARN cluster. The master also recovers the jobs in the case of a failure.
- The job starts processing the data and produces results to external systems (e.