presto on redis

Presto on redis 

PRESTO的安装和配置

1 安装

1 下载presto包

(1)

wget https://repo1.maven.org/maven2/com/facebook/presto/presto-server/0.179/presto-server-0.179.tar.gz

2 上传presto包上传到服务器

将（1）上传至10.118.242.72  10.118.242.75 10.118.242.76  /usr/local/下

3 解压presto包

tar –zxvf /usr/local/presto-server-0.179.tar.gz

2 配置presto

在3台机器/usr/local/presto-server-0.179下创建etc目录

共创建4个配置文件

node.properties

 

node.environment=production

node.id=72 #其他机器相应修改

node.data-dir=/data/presto/data

 

config.properties (72)

 

coordinator=true

node-scheduler.include-coordinator=false

http-server.http.port=8185

query.max-memory=50GB

query.max-memory-per-node=1GB

discovery-server.enabled=true

discovery.uri=http://10.118.242.72:8185

 

config.properties (75/76)

 

coordinator=false

http-server.http.port=8185

query.max-memory=50GB

query.max-memory-per-node=1GB

discovery.uri=http://10.118.242.72:8185

 

jvm.config

 

-server

-Xmx16G

-XX:+UseG1GC

-XX:G1HeapRegionSize=32M

-XX:+UseGCOverheadLimit

-XX:+ExplicitGCInvokesConcurrent

-XX:+HeapDumpOnOutOfMemoryError

 

log.properties

 

com.facebook.presto=INFO

3 参考文档

https://prestodb.io/docs/current/index.html

Redis connnector的安装和配置

1 配置

在3台机器的/usr/local/presto-server-0.179/etc/下创建catalog/redis.properties文件

 

connector.name=redis

redis.table-names=redis.cytest

redis.nodes=10.118.242.75:6379   #按照实际情况配置

redis.hide-internal-columns=false

redis.password=sfdp

 

在3台机器的/usr/local/presto-server-0.179/etc/下创建redis/cytest.json文件

 

{

   "tableName": "cytest",

   "schemaName": "redis",

   "key": {

       "dataFormat": "raw",

       "fields": [

            {

                "name":"redis_key",

                "dataFormat":"_default",

                "type":"VARCHAR",

                "hidden":"false"

           }

       ]

   },

   "value": {

       "dataFormat": "json",

       "fields": [

           {

                "name":"row_number",

                "mapping":"rowNumber",

                "type":"BIGINT"

           },

           {

                "name":"customer_key",

                "mapping":"customerKey",

                "type":"BIGINT"

           },

           {

                "name":"name",

                "mapping":"name",

                "type":"VARCHAR"

           }

      ]

    }

}

Presto onredis的例子

1 启动presto

分别在3台机器启动presto server

/usr/local/presto-server-0.179/bin/launcherstart

日志在/data/presto/data/var/log/server.log

2 在redis插入数据

set key1 "{\"rowNumber\":1,\"customerKey\":1,\"name\":\"Customer#000000001\"}"

 setkey2"{\"rowNumber\":3,\"customerKey\":3,\"name\":\"Customer#000000003\"}"

 setkey3"{\"rowNumber\":5,\"customerKey\":5,\"name\":\"Customer#000000005\"}"

 setkey4 "{\"rowNumber\":7,\"customerKey\":7,\"name\":\"Customer#000000007\"}"

 setkey5"{\"rowNumber\":9,\"customerKey\":9,\"name\":\"Customer#000000009\"}"

 setkey6"{\"rowNumber\":9,\"customerKey\":19,\"name\":\"Customer#0000000019\"}"

3 在presto-cli查询

下载presto-cli

wget https://repo1.maven.org/maven2/com/facebook/presto/presto-cli/0.179/presto-cli-0.179-executable.jar

将jar文件放在/usr/local/presto-server-0.179/bin/

执行

/usr/local/presto-server-0.179/bin/presto-cli-0.179-executable.jar--server 10.118.242.72:8185 --catalog redis --schema redis

 

写入sql