broker druid 查询_即时查询工具| Druid

Druid是一个快速的列式分布式的支持实时分析的数据存储系统，在处理PB级别数据、毫秒级查询、数据实时处理方面，比传统的OLAP系统有了显著的性能改进。

Druid数据结构

与Druid架构相辅相成的是其基于DataSource与Segment的数据结构，它们共同成就了Druid的高性能优势。

Druid安装(单机版)

安装部署

imply集成了Druid，提供了Druid从部署到配置到各种可视化工具的完整的解决方案，imply有点类似于我们之前学过的Cloudera Manager

1.解压tar -zxvf imply-2.7.10.tar.gz -C /opt/module

目录说明如下： - bin/ - run scripts forincluded software.- conf/ - template configurations fora clustered setup.- conf-quickstart/*- configurations for the single-machine quickstart.

- dist/ - all included software.

- quickstart/ - files related to the single-machine quickstart.

2.修改配置文件 1)修改Druid的ZK配置

[kris@hadoop101 _common]$ vim /opt/module/imply/conf/druid/_common/common.runtime.properties

druid.zk.service.host=hadoop101:2181,hadoop102:2181,hadoop103:2181

2)修改启动命令参数，使其不校验不启动内置ZK

[kris@hadoop101 supervise]$ vim /opt/module/imply/conf/supervise/quickstart.conf

:verify bin/verify-java

#:verify bin/verify-default-ports

#:verify bin/verify-version-check

:kill-timeout 10

#!p10 zk bin/run-zk conf-quickstart

3.启动1)启动zookeeper2)启动imply

[kris@hadoop101 imply-2.7.10]$ bin/supervise -c conf/supervise/quickstart.conf

[Fri Jan3 13:37:51 2020] Running command[coordinator], logging to[/opt/module/imply-2.7.10/var/sv/coordinator.log]: bin/run-druid coordinator conf-quickstart

[Fri Jan3 13:37:51 2020] Running command[broker], logging to[/opt/module/imply-2.7.10/var/sv/broker.log]: bin/run-druid broker conf-quickstart

[Fri Jan3 13:37:51 2020] Running command[historical], logging to[/opt/module/imply-2.7.10/var/sv/historical.log]: bin/run-druid historical conf-quickstart

[Fri Jan3 13:37:51 2020] Running command[overlord], logging to[/opt/module/imply-2.7.10/var/sv/overlord.log]: bin/run-druid overlord conf-quickstart

[Fri Jan3 13:37:51 2020] Running command[middleManager], logging to[/opt/module/imply-2.7.10/var/sv/middleManager.log]: bin/run-druid middleManager conf-quickstart

[Fri Jan3 13:37:51 2020] Running command[imply-ui], logging to[/opt/module/imply-2.7.10/var/sv/imply-ui.log]: bin/run-imply-ui-quickstart conf-quickstart

说明：每启动一个服务均会打印出一条日志。可以通过/opt/module/imply-2.7.10/var/sv/查看服务启动时的日志信息

[kris@hadoop101 sv]$ ll

总用量904

-rw-rw-r-- 1 kris kris 77528 1月 3 13:38broker.log-rw-rw-r-- 1 kris kris 588001 1月 3 14:07coordinator.log-rw-rw-r-- 1 kris kris 76857 1月 3 13:38historical.log-rw-rw-r-- 1 kris kris 11599 1月 3 14:06 imply-ui.log-rw-rw-r-- 1 kris kris 70725 1月 3 13:38middleManager.log-rw-rw-r-- 1 kris kris 93658 1月 3 14:07overlord.log

查看端口号9095的启动情况：

[kris@hadoop101 sv]$ netstat-anp | grep 9095(Not all processes could be identified, non-owned process infowill not be shown, you would have to be root to see it all.)

tcp0 0 :::9095 :::* LISTEN 6118/imply-ui-linux

tcp0 0 ::ffff:192.168.1.101:9095 ::ffff:192.168.1.1:54884 TIME_WAIT -tcp0 0 ::ffff:192.168.1.101:9095 ::ffff:192.168.1.1:54883 TIME_WAIT -

登录hadoop101:9095查看

停止服务

按Ctrl + c中断监督进程，如果想中断服务后进行干净的启动，请删除/opt/module/imply-2.7.10/var/目录。

从hadoop加载数据

加载数据

批量摄取维基百科样本数据，文件位于quickstart/wikipedia-2016-06-27-sampled.json。使用quickstart/wikipedia-index-hadoop.json 摄取任务文件。

bin/post-index-task --file quickstart/wikipedia-index-hadoop.json

此命令将启动Druid Hadoop摄取任务。    摄取任务完成后，数据将由历史节点加载，并可在一两分钟内进行查询。

[kris@hadoop101 imply]$ bin/post-index-task --file quickstart/wikipedia-index-hadoop.json

Beginning indexing dataforwikipedia

Task started: index_hadoop_wikipedia_2020-02-08T09:25:37.204Z

Task log: http://localhost:8090/druid/indexer/v1/task/index_hadoop_wikipedia_2020-02-08T09:25:37.204Z/log

Task status: http://localhost:8090/druid/indexer/v1/task/index_hadoop_wikipedia_2020-02-08T09:25:37.204Z/status

Task index_hadoop_wikipedia_2020-02-08T09:25:37.204Z still running...

Task index_hadoop_wikipedia_2020-02-08T09:25:37.204Z still running...

Task index_hadoop_wikipedia_2020-02-08T09:25:37.204Z still running...

Task index_hadoop_wikipedia_2020-02-08T09:25:37.204Z still running...

Task finished with status: SUCCESS

Completed indexing dataforwikipedia. Now loading indexed data onto the cluster...

wikipedia loading complete! You may now query your data

SQL查询

select * from wikipedia

[kris@hadoop101 imply]$ curl -XPOST -H'Content-Type: application/json' -d @quickstart/wikipedia-kafka-supervisor.json http://hadoop101:8090/druid/indexer/v1/supervisor

{"id":"wikipedia-kafka"}

加载实时数据：

[kris@hadoop101 imply]$ curl -O https://static.imply.io/quickstart/wikiticker-0.4.tar.gz

% Total % Received %Xferd Average Speed Time Time Time Current

Dload Upload Total Spent Left Speed100 30.0M 100 30.0M 0 0 535k 0 0:00:57 0:00:57 --:--:-- 623k