大数据通用系统解决方案(分别基于Open,AWS,Aliyun,和三者的使用技术)
应用场景
物联网领域、车联网、智能城市,智能生活
MQTT | 端和云之间的通信 | |||||
1 open - EMQ (百万级分布式开源物联网MQTT消息服务器) | ||||||
2 aws - AWS IoT | ||||||
3 ali - Aliyun IOT (包括 Aliyun MQTT) | ||||||
消息中间件 | ||||||
MQ | 1 open - Kafka | |||||
2 aws - Kinesis | ||||||
3 ali - RocketMQ | ||||||
Infrastructure | ||||||
infra | 1 open - Ambari(Hotonworks) | |||||
基盘搭建 | 2 aws - aws-emr | |||||
3 ali - E-MapReduce | ||||||
计算 | ||||||
Realtime Compute | 1 open - Spark Streaming,Flink | |||||
即时计算 | 2 aws - 暂无 | |||||
3 ali - 阿里云实时计算(Alibaba Cloud Realtime Compute),基于Flink | ||||||
Batch Compute | 1 open - Spark Batch, Hadoop MR | |||||
离线计算 | 2 aws - AWS Batch | |||||
3 ali - BatchCompute | ||||||
分析 | ||||||
Analysis | 1 open - Apache Kylin™等 | |||||
分析 | 2 aws - Amazon QuickSight等 | |||||
3 ali - Quick BI,画像分析等 | ||||||
存储 | ||||||
File Store | 1 open - HDFS | |||||
文件存储 | 2 aws - FSx | |||||
3 ali - NAS | ||||||
Object Store | 1 open - Swift | |||||
对象存储 | 2 aws - S3 | |||||
3 ali - OSS | ||||||
NoSql DB | 1 open - HBase | |||||
2 aws - DynamoDB | ||||||
3 ali - OceanBase | ||||||
缓存 | ||||||
Data Cache | 1 open - Redis,Memcached | |||||
缓存 | 2 aws - Amazon ElastiCache(Redis 和 Memcached 引擎之上) | |||||
3 ali - 阿里云数据库 Redis 版(ApsaraDB for Redis)是兼容开源 Redis 协议的 Key-Value 类型在线存储服务 | ||||||
搜索 | ||||||
Data Search | 1 open - Elasticsearch | |||||
搜索引擎 | 2 aws - Amazon ES (基于开源Elasticsearch一种常见的开源搜索和分析引擎) | |||||
3 ali - 阿里云Elasticsearch,提供基于开源Elasticsearch服务 | ||||||
数据仓库 | ||||||
data warehousing | 1 open - Hive | |||||
数据仓库 | 2 aws - Amazon Redshift | |||||
3 ali - Data Lake Analytics |