用过presto的同学都知道,客服端访问presto集群,其实是通过http协议访问的presto的coordinator节点;大多数情况是没啥问题;可是服务器直接暴露给用户,或多或少还是有些安全顾虑;所以我们在客户端和presto集群之间加了一层访问代理,如下图所示
所有的用户请求通过代理访问presto集群;过程如下
1.客户端可以使用jdbc,Python等方式访问代理,使用方式和直接访问集群方式一样,只不过以前的host是coordinator,而现在是presot-proxy
2.代理获得客户端的请求后封装post请求访问presto集群,presto集群响应请求的报文如下,代理获得presto集群的响应报文后,返回给客户端
{
"id":"20181107_132829_00014_u28jg",
"infoUri":"http://192.168.120.4:8099/ui/query.html?20181107_132829_00014_u28jg",
"nextUri":"http://192.168.120.4:8099/v1/statement/20181107_132829_00014_u28jg/1",
"stats":{
"state":"QUEUED",
"queued":true,
"scheduled":false,
"nodes":0,
"totalSplits":0,
"queuedSplits":0,
"runningSplits":0,
"completedSplits":0,
"cpuTimeMillis":0,
"wallTimeMillis":0,
"queuedTimeMillis":0,
"elapsedTimeMillis":0,
"processedRows":0,
"processedBytes":0,
"peakMemoryBytes":0
}
}
3.客户端获得报文后,根据报文中nextUri直接发送GET请求给presto集群的coordinator节点
4.coordinator节点负责分发任务给node,计算并返回结果
我们从如上过程,可以看到,其实presto-proxy只是处理client的第一次请求,并返回报文;剩下的所有请求都是client根据报文中的nextUri直接发送GET请求给presto集群;对于用户来说,他们能看到的只是步骤1,其他的步骤对用户来说都是透明的
现在来实现一下这个proxy,其实很简单,核心代码也就十几行
package com.fan.prestoproxy.controller;
import org.apache.commons.io.IOUtils;
import org.apache.http.HttpResponse;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.DefaultHttpClient;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.ResponseBody;
import javax.servlet.http.HttpServletRequest;
import java.io.BufferedReader;
import java.io.InputStream;
import java.util.Calendar;
import java.util.Enumeration;
import java.util.Locale;
@Controller
@RequestMapping("/v1")
public class QueryController {
@ResponseBody
@RequestMapping(value= "statement",method = RequestMethod.POST,produces="application/json;charset=UTF-8")
public String getResponse(HttpServletRequest request ) throws Exception {
DefaultHttpClient hc = new DefaultHttpClient(); //初始化一个HTTP的客户端对象
HttpPost httppost = new HttpPost("http://192.168.120.4:8099/v1/statement");
BufferedReader reader = request.getReader();
String statement = reader.readLine();
httppost.setHeader("X-Presto-Source", request.getHeader("x-presto-source"));
httppost.setHeader("X-Presto-User", request.getHeader("x-presto-user"));
httppost.setHeader("User-Agent", request.getHeader("user-agent"));
httppost.setHeader("X-Presto-Time-Zone", Calendar.getInstance().getTimeZone().getID());
httppost.setHeader("X-Presto-Language", Locale.CHINA.getLanguage());
Enumeration<String> headerNames = request.getHeaderNames();
while(headerNames.hasMoreElements()){
String key = headerNames.nextElement();
System.out.println(key+" "+request.getHeader(key));
}
httppost.setEntity(new StringEntity(statement));
HttpResponse httpresponse = hc.execute(httppost);
InputStream is = httpresponse.getEntity().getContent();
String body = IOUtils.toString(is, "utf-8");
System.out.println(body);
return body;
}
}
http://192.168.120.4:8099/v1/statement
也就是presto集群对外提供的接口;测试代码如下,就是普通的jdbc代码,只不过jdbc的host指向的是proxy的host
public static void jdbc()throws Exception{
Class.forName("com.facebook.presto.jdbc.PrestoDriver");
String url = "jdbc:presto://localhost:8080";
Connection connection = DriverManager.getConnection(url, "user", null);
Statement statement = connection.createStatement();
long begin = System.currentTimeMillis();
String sql = "select * from mysql.shiro.sh_user";
ResultSet rs = statement.executeQuery(sql) ;
long end = System.currentTimeMillis();
System.out.println((end - begin) + " ms ");
while(rs.next()){
System.out.println(rs.getString("name")+ "," + rs.getString("passwd"));
}
statement.close();
connection.close();
}
代理的作用不局限于此,其实在代理层我们可以实现自己的数据访问权限,如果用户的请求不合规,我们也可以直接在代理层抛出异常
end