在streaming process中为什么需要类似sql查询语言

从现有的CEP(Complex Event Processing)框架,像强大得Esper,我们学到很多工作完全可以通过写sql来完成的,不需要编写任何的代码。所以对于流处理我认为也需要这样的类似sql查询语言。目前像Storm这样的流处理平台框架强迫使用者去实现像silding window,一些query pattern的操作。我认为并不是每一个案例开发者都喜欢重写这些东西。

所以我们需要high level的抽象,我可以完全把一些算法封装好,直接拼装调用即可,或者直接支持sql语法。我们可以从Hive for HadoopSpark SQLapache Flink中学习到很多。

下面列举一下为什么需要类似sql查询语言的原因:
1. 实时分析很复杂,业务开发人员很不情愿去实现一些像sliding window,或者时间事件模式的算法;
2. SQL简单,很容易理解和学习,开发成本低;
3. SQL语言简短通用,而且快
4. SQL自身的核心语法可以解决90%的问题
5. 历年来BI专家倾向的语言
6. 实时分析可以更好的优化SQL语言模型的执行计划,大部分的优化操作已经在研究,有些已经研究出来了可以直接拿过来用。

Why We need SQL like Query Language for Realtime Streaming Analytics?

注:Stream Processing with Apache Flink网页版 Book Description With Early Release ebooks, you get books in their earliest form—the author's raw and unedited content as he or she writes—so you can take advantage of these technologies long before the official release of these titles. You’ll also receive updates when significant changes are made, new chapters are available, and the final ebook bundle is released. Get started with Apache Flink, the open source framework that enables you to process streaming data—such as user interactions, sensor data, and machine logs—as it arrives. With this practical guide, you’ll learn how to use Apache Flink’s stream processing APIs to implement, continuously run, and maintain real-world applications. Authors Fabian Hueske, one of Flink’s creators, and Vasia Kalavri, a core contributor to Flink’s graph processing API (Gelly), explains the fundamental concepts of parallel stream processing and shows you how streaming analytics differs from traditional batch data analysis. Software engineers, data engineers, and system administrators will learn the basics of Flink’s DataStream API, including the structure and components of a common Flink streaming application. Solve real-world problems with Apache Flink’s DataStream API Set up an environment for developing stream processing applications for Flink Design streaming applications and migrate periodic batch workloads to continuous streaming workloads Learn about windowed operations that process groups of records Ingest data streams into a DataStream application and emit a result stream into different storage systems Implement stateful and custom operators common in stream processing applications Operate, maintain, and update continuously running Flink streaming applications Explore several deployment options, including the setup of highly available installations
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值