Ambari安装Presto

标签： Presto

2018年04月15日 20:12:48 18人阅读评论(0) 收藏举报

分类：

Ambari（2）

作者同类文章 X

Presto

结点规划

结点	角色
node1	Presto coordinator
node2	Presto worker与Client
node3	Presto worker与Client
node4	Ambari Server

禁用requiretty

安装presto的结点需要禁用requiretty,编辑/etc/sudoers文件，注释掉Defaults requiretty

下载ambari-presto-service

[root@node4 ~]# wget https://github.com/prestodb/ambari-presto-service/releases/download/v1.2/ambari-presto-1.2.tar.gz

[root@node4 ~]# mkdir /var/lib/ambari-server/resources/stacks/HDP/2.6/services/PRESTO

[root@node4 ~]# tar xvf ambari-presto-1.2.tar.gz --strip-components 1 -C /var/lib/ambari-server/resources/stacks/HDP/2.6/services/PRESTO

[root@node4 ~]# chmod -R +x /var/lib/ambari-server/resources/stacks/HDP/2.6/services/PRESTO/*

下载presto离线安装包并配置仓库

在线安装太慢，这里下载presto离线安装包，并配置仓库。Presto coordinator结点、Presto worker与Client均从该仓库下载包。

下载presto离线安装包

[root@node4 ~]# mkdir presto_repo

[root@node4 ~]# cd presto_repo/

[root@node4 presto_repo]# wget http://search.maven.org/remotecontent?filepath=com/facebook/presto/presto-server-rpm/0.198/presto-server-rpm-0.198.rpm

[root@node4 presto_repo]# wget http://search.maven.org/remotecontent?filepath=com/facebook/presto/presto-cli/0.198/presto-cli-0.198-executable.jar

配置仓库

[root@node4 presto_repo]# ls
    presto-cli-0.198-executable.jar  presto-server-rpm-0.198.rpm

[root@node4 presto_repo]# yum install createrepo

[root@node4 presto_repo]# createrepo ${PWD}

[root@node4 presto_repo]# python -m SimpleHTTPServer 8081

安装presto的结点增加presto仓库

node1结点要安装Presto coordinator，需要配置presto仓库
[root@node1 ~]# cat /etc/yum.repos.d/presto.repo
    [presto_repo]
    name=presto_repo
    baseurl=http://node4:8081
    enable=1
    gpgcheck=0

node2结点要安装Presto worker与Client，需要配置presto仓库
[root@node1 ~]# scp /etc/yum.repos.d/presto.repo root@node2:/etc/yum.repos.d

node3结点要安装Presto worker与Client，需要配置presto仓库
[root@node1 ~]# scp /etc/yum.repos.d/presto.repo root@node3:/etc/yum.repos.d

修改presto版本和安装包下载路径

[root@node4 ~]# vim /var/lib/ambari-server/resources/stacks/HDP/2.6/services/PRESTO/metainfo.xml
        <version>0.198</version>

[root@node4 ~]# vim /var/lib/ambari-server/resources/stacks/HDP/2.6/services/PRESTO/package/scripts/download.ini       
        [download]
        presto_rpm_url = http://node4:8081/presto-server-rpm-0.198.rpm
        presto_cli_url = http://node4:8081/presto-cli-0.198-executable.jar

Ambari安装Presto

[root@node4 ~]# ambari-server restart

按Ambari正常安装服务即可完成安装。discovery.uri:http://node1:8285

Presto查询Hive和Kafka数据

添加Hive和Kafka Connectors

Ambari=>Presto=>Configs=>Connectors添加如下Connectors

{
  'hive':[
    'connector.name=hive-hadoop2',
    'hive.metastore.uri=thrift://node2:9083' #Hive Metastore 
  ],
  'kafka':[
    'connector.name=kafka',
    'kafka.table-names=test',#kafka topic
    'kafka.nodes=node2:6667' #kafka bootstrap_servers
  ]
}

Presto查询Hive

[root@node3 ~]# /usr/lib/presto/bin/presto-cli --server node1:8285 --catalog hive --schema default

presto:default> select departmentid,classid,sum(language) as language from test group by departmentid,classid;

Presto查询Kafka

[root@node3 ~]# /usr/lib/presto/bin/presto-cli --server node1:8285 --catalog kafka --schema default

presto:default> select json_extract_scalar(_message, '$.language') as language from test limit 3;