Drill的安装

最新推荐文章于 2022-12-06 14:42:46 发布

Xlucas

最新推荐文章于 2022-12-06 14:42:46 发布

阅读量1.5k

点赞数

分类专栏： SQL on hadoop 文章标签： drill sql

本文链接：https://blog.csdn.net/paicMis/article/details/52714294

版权

SQL on hadoop 专栏收录该内容

9 篇文章 0 订阅

订阅专栏

1、解压

[root@master opt]# tar -zxvf apache-drill-1.7.0.tar.gz 
[root@master opt]# ls
apache-drill-1.7.0  apache-drill-1.7.0.tar.gz  hadoop-2.6.2

2.启动

[hadoop@master bin]$ ./drill-embedded 
Invalid initial heap size: -Xms4G
The specified size exceeds the maximum representable size.
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

出现了内存不足的情况需求修改 conf 目录下面的 drill-env.sh 文件
DRILL_MAX_DIRECT_MEMORY=”512M”
DRILL_HEAP=”512M
再次启动drill
[root@master bin]# ./drill-embedded
Jul 11, 2016 8:20:14 AM org.glassfish.jersey.server.ApplicationHandler initialize
INFO: Initiating Jersey application, version Jersey: 2.8 2014-04-29 01:25:26…
apache drill 1.7.0
“json ain’t no thang”
0: jdbc:drill:zk=local>

测试drill

0: jdbc:drill:zk=local> select * from dfs.`/opt/apache-drill-1.7.0/sample-data/nation.parquet`;

SLF4J: Failed to load class “org.slf4j.impl.StaticLoggerBinder”.
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
+————–+—————–+————–+———————–+
| N_NATIONKEY | N_NAME | N_REGIONKEY | N_COMMENT |
+————–+—————–+————–+———————–+
| 0 | ALGERIA | 0 | haggle. carefully f |
| 1 | ARGENTINA | 1 | al foxes promise sly |
| 2 | BRAZIL | 1 | y alongside of the p |
| 3 | CANADA | 1 | eas hang ironic, sil |
| 4 | EGYPT | 4 | y above the carefull |
| 5 | ETHIOPIA | 0 | ven packages wake qu |
| 6 | FRANCE | 3 | refully final reques |
| 7 | GERMANY | 3 | l platelets. regular |
| 8 | INDIA | 2 | ss excuses cajole sl |
| 9 | INDONESIA | 2 | slyly express asymp |
| 10 | IRAN | 4 | efully alongside of |
| 11 | IRAQ | 4 | nic deposits boost a |
| 12 | JAPAN | 2 | ously. final, expres |
| 13 | JORDAN | 4 | ic deposits are blit |
| 14 | KENYA | 0 | pending excuses hag |
| 15 | MOROCCO | 0 | rns. blithely bold c |
| 16 | MOZAMBIQUE | 0 | s. ironic, unusual a |
| 17 | PERU | 1 | platelets. blithely |
| 18 | CHINA | 2 | c dependencies. furi |
| 19 | ROMANIA | 3 | ular asymptotes are |
| 20 | SAUDI ARABIA | 4 | ts. silent requests |
| 21 | VIETNAM | 2 | hely enticingly expr |
| 22 | RUSSIA | 3 | requests against th |
| 23 | UNITED KINGDOM | 3 | eans boost carefully |
| 24 | UNITED STATES | 1 | y final packages. sl |
+————–+—————–+————–+———————–+
25 rows selected (4.292 seconds)

访问HDFS上面的文件
1、登录 ui http://192.168.18.144:8047/
2、添加 Storage Plugins

{
  "type": "file",
  "enabled": true,
  "connection": "hdfs://master:9000/",
  "config": null,
  "workspaces": {
    "root": {
      "location": "/opt/apache-drill-1.7.0",
      "writable": true,
      "defaultInputFormat": null
    }
  },
  "formats": {
    "csv": {
      "type": "text",
      "extensions": [
        "csv"
      ],
      "delimiter": ","
    },
    "tsv": {
      "type": "text",
      "extensions": [
        "tsv"
      ],
      "delimiter": "\t"
    },
    "parquet": {
      "type": "parquet"
    }
  }
}

访问hdfs上面的文件

0: jdbc:drill:zk=local> select * from hdfs.`/tmp/nation.parquet`;

退出连接

0: jdbc:drill:zk=local> !quit;

Closing: org.apache.drill.jdbc.impl.DrillConnectionImpl

分布式的安装和测试
1、修改drill-override.conf 配置zookeeper地址，我这里指配置一台地址

[root@master conf]# cat drill-override.conf 
# See 'drill-override-example.conf' for example configurations
drill.exec: {
  cluster-id: "drillbits1",
  zk.connect: "master:2181"
}

2、将drill部分到其他节点上去
scp -r /opt/apache-drill-1.7.0/ slave2:/opt/
3、在每台机器上面启动drill ，drill不像其他的一样有start-all，必须在每台机器上面去执行

[root@master bin]# ./drillbit.sh start
starting drillbit, logging to /opt/apache-drill-1.7.0/log/drillbit.out

启动以后我们可以在ui上面看到3台机器

测试分布式的drill
./sqlline -u jdbc:drill:zk=master 通过这个启动一个客户端

[root@slave1 bin]# ./sqlline -u jdbc:drill:zk=master
apache drill 1.7.0 
"just drill it"
0: jdbc:drill:zk=master> select * from hdfs.`/tmp/nation.parquet`;

+————–+—————–+————–+———————–+
| N_NATIONKEY | N_NAME | N_REGIONKEY | N_COMMENT |
+————–+—————–+————–+———————–+
| 0 | ALGERIA | 0 | haggle. carefully f |
| 1 | ARGENTINA | 1 | al foxes promise sly |
| 2 | BRAZIL | 1 | y alongside of the p |
| 3 | CANADA | 1 | eas hang ironic, sil |
| 4 | EGYPT | 4 | y above the carefull |
| 5 | ETHIOPIA | 0 | ven packages wake qu |
| 6 | FRANCE | 3 | refully final reques |
| 7 | GERMANY | 3 | l platelets. regular |
| 8 | INDIA | 2 | ss excuses cajole sl |
| 9 | INDONESIA | 2 | slyly express asymp |
| 10 | IRAN | 4 | efully alongside of |
| 11 | IRAQ | 4 | nic deposits boost a |
| 12 | JAPAN | 2 | ously. final, expres |
| 13 | JORDAN | 4 | ic deposits are blit |
| 14 | KENYA | 0 | pending excuses hag |
| 15 | MOROCCO | 0 | rns. blithely bold c |
| 16 | MOZAMBIQUE | 0 | s. ironic, unusual a |
| 17 | PERU | 1 | platelets. blithely |
| 18 | CHINA | 2 | c dependencies. furi |
| 19 | ROMANIA | 3 | ular asymptotes are |
| 20 | SAUDI ARABIA | 4 | ts. silent requests |
| 21 | VIETNAM | 2 | hely enticingly expr |
| 22 | RUSSIA | 3 | requests against th |
| 23 | UNITED KINGDOM | 3 | eans boost carefully |
| 24 | UNITED STATES | 1 | y final packages. sl |
+————–+—————–+————–+———————–+
25 rows selected (4.121 seconds)