人一部分的愤怒来自于无法接受想到和做到之间的巨大差距!!!
本篇文章主要是介绍如何基于okhttp来实现网络请求的监控,可能很多同学要问,为啥要监控?监控主要的目标就是真实呈现和还原线上用户的使用状况,为应用整体优化提供指标和方向。平常我们在开发环境中普遍网络状况还是很好的,不能很好的反应出线上用户真实的使用情况。而加入监控后,可以从数据中发现用户的真实使用情况,并拆分请求过程中每个阶段的耗时和失败原因,然后再给予相关指标来进行优化(把锅甩出去)。
了解OkHttp
因为是基于OkHttp来实现监控,所以我们要稍微了解下OkHttp的实现,这里我们了解两个方面就行。
一次网络请求经历了哪些过程
通过域名访问的方式来请求网络时,会经历下列过程:
- DNS解析:通过域名服务器或者本地host将域名解析成ip地址
- 建立连接:三次握手
- 发送数据:通过GET/POST/PUT等方式将数据(header和body)发送给服务器,
- 接受数据:接受服务器返回数据:响应头和body
- 断开链接:四次挥手断开链接
OkHttp库实现了哪些网络请求过程的状态回调
如上图所示,该图来自于OkHttp的Wiki文档,每个过程还是比较清晰的,简单解释下
- callStart:一次请求开始了
- dns:dns解析过程
- connectStart: 开始建立连接了
- secureConnect: 开始建立TSL安全链接
- connectEnd: 链接建立结束:可能建立失败,失败后可以重试
- requestHeaders:发送请求头
- requestBody:发送请求body
- responseHeaders:客户端接受响应头
- responseBody:客户端接受响应body
- connectionReleased:链接释放
- callEnd:一次请求结束
那么上述流程如何反馈到代码上呢?关键类:EventListener
public abstract class EventListener {
public static final EventListener NONE = new EventListener() {
};
static EventListener.Factory factory(final EventListener listener) {
return new EventListener.Factory() {
public EventListener create(Call call) {
return listener;
}
};
}
public void callStart(Call call) {
}
public void dnsStart(Call call, String domainName) {
}
public void dnsEnd(Call call, String domainName, List<InetAddress> inetAddressList) {
}
public void connectStart(Call call, InetSocketAddress inetSocketAddress, Proxy proxy) {
}
public void secureConnectStart(Call call) {
}
public void secureConnectEnd(Call call, @Nullable Handshake handshake) {
}
public void connectEnd(Call call, InetSocketAddress inetSocketAddress, Proxy proxy,
@Nullable Protocol protocol) {
}
public void connectFailed(Call call, InetSocketAddress inetSocketAddress, Proxy proxy,
@Nullable Protocol protocol, IOException ioe) {
}
public void connectionAcquired(Call call, Connection connection) {
}
public void connectionReleased(Call call, Connection connection) {
}
public void requestHeadersStart(Call call) {
}
public void requestHeadersEnd(Call call, Request request) {
}
public void requestBodyStart(Call call) {
}
public void requestBodyEnd(Call call, long byteCount) {
}
public void responseHeadersStart(Call call) {
}
public void responseHeadersEnd(Call call, Response response) {
}
public void responseBodyStart(Call call) {
}
public void responseBodyEnd(Call call, long byteCount) {
}
public void callEnd(Call call) {
}
public void callFailed(Call call, IOException ioe) {
}
public interface Factory {
EventListener create(Call call);
}
}
上述EventListener对象是如何和请求对象RealCall建立联系的呢?
//RealCall.java
static RealCall newRealCall(OkHttpClient client, Request originalRequest, boolean forWebSocket) {
// Safely publish the Call instance to the EventListener.
RealCall call = new RealCall(client, originalRequest, forWebSocket);
call.eventListener = client.eventListenerFactory().create(call);
return call;
}
OkHttpClient.java
public Builder eventListenerFactory(EventListener.Factory eventListenerFactory) {
if (eventListenerFactory == null) {
throw new NullPointerException("eventListenerFactory == null");
}
this.eventListenerFactory = eventListenerFactory;
return this;
}
从代码可以看出,实际上可以在创建OkHttpClient对象时指定全局的EventListener对应的工厂累Factory。现在知道了回调和接入实际,就可以动手了。
实现监控
话不多说,直接上代码把:
OkHttpClient.Builder httpClientBuilder = new OkHttpClient.Builder();
CallbackBuilder callbackBuilder = new CallbackBuilder();
Factory factory =
new SuHttpEventListener.XYFactory(callbackBuilder.factory, callbackBuilder.stepCallback);
httpClientBuilder.eventListenerFactory(factory);
public class SuHttpEventListener extends EventListener {
public static final String TAG = "SuHttpEventListener";
private static final Charset UTF8 = Charset.forName("UTF-8");
public static final int TIME_OUT_MILLS = 60 * 1000;
private final long callStartNanos;
private long dnsStartNano;
private long connectStartNano;
private long requestStartNano;
private EventListener subEventListener;
private SuKVEventListener stepCallback;
private HttpData mHttpData = new HttpData();
/**
* 套一层的Factory
*/
static class XYFactory implements Factory {
private Factory subFactory;
private SuKVEventListener mStepCallback;
XYFactory(Factory subFactory, SuKVEventListener stepCallback) {
this.subFactory = subFactory;
this.mStepCallback = stepCallback;
}
@Override public EventListener create(Call call) {
EventListener subListener = null;
if (subFactory != null) {
subListener = subFactory.create(call);
}
if (SuHttpMonitor.checkIsLocalHost(call.request().url().host())) {
return new SuHttpEventListener(subListener, null);
}
return new SuHttpEventListener(subListener, mStepCallback);
}
}
SuHttpEventListener(EventListener subEventListener, SuKVEventListener stepCallback) {
this.subEventListener = subEventListener;
this.stepCallback = stepCallback;
this.callStartNanos = System.nanoTime();
}
@Override public void dnsStart(Call call, String domainName) {
mHttpData.stepCode = HttpEventStep.dnsStart;
if (subEventListener != null) {
subEventListener.dnsStart(call, domainName);
}
dnsStartNano = System.nanoTime();
}
@Override public void dnsEnd(Call call, String domainName, List<InetAddress> inetAddressList) {
if (subEventListener != null) {
subEventListener.dnsEnd(call, domainName, inetAddressList);
}
if (dnsStartNano <= 0) {
return;
}
long cost = getCost(dnsStartNano);
if (cost < 0) {
return;
}
if (stepCallback != null) {
mHttpData.dnsCost = cost;
}
dnsStartNano = 0;
}
@Override public void connectStart(Call call, InetSocketAddress inetSocketAddress, Proxy proxy) {
mHttpData.stepCode = HttpEventStep.connectStart;
if (subEventListener != null) {
subEventListener.connectStart(call, inetSocketAddress, proxy);
}
connectStartNano = System.nanoTime();
}
@Override public void secureConnectStart(Call call) {
mHttpData.stepCode = HttpEventStep.secureConnectStart;
if (subEventListener != null) {
subEventListener.secureConnectStart(call);
}
}
@Override public void secureConnectEnd(Call call, Handshake handshake) {
if (subEventListener != null) {
subEventListener.secureConnectEnd(call, handshake);
}
}
@Override public void connectEnd(Call call, InetSocketAddress inetSocketAddress, Proxy proxy,
Protocol protocol) {
if (subEventListener != null) {
subEventListener.connectEnd(call, inetSocketAddress, proxy, protocol);
}
if (connectStartNano <= 0) {
return;
}
long cost = getCost(connectStartNano);
if (cost <= 0) {
return;
}
if (stepCallback != null) {
mHttpData.proxy = proxy == null ? null : proxy.toString();
mHttpData.inetSocketAddress =
inetSocketAddress == null ? null : inetSocketAddress.toString();
mHttpData.protocol = protocol == null ? null : protocol.toString();
mHttpData.connectCost = cost;
}
}
@Override public void connectionAcquired(Call call, Connection connection) {
mHttpData.stepCode = HttpEventStep.connectionAcquired;
if (subEventListener != null) {
subEventListener.connectionAcquired(call, connection);
}
requestStartNano = System.nanoTime();
}
@Override public void connectionReleased(Call call, Connection connection) {
log("connectionReleased");
if (subEventListener != null) {
subEventListener.connectionReleased(call, connection);
}
if (requestStartNano <= 0) {
return;
}
long cost = getCost(requestStartNano);
if (cost <= 0) {
return;
}
if (stepCallback != null) {
mHttpData.responseCost = cost;
}
requestStartNano = 0;
}
@Override public void connectFailed(Call call, InetSocketAddress inetSocketAddress, Proxy proxy,
Protocol protocol, IOException ioe) {
super.connectFailed(call, inetSocketAddress, proxy, protocol, ioe);
log("connectFailed");
if (subEventListener != null) {
subEventListener.connectFailed(call, inetSocketAddress, proxy, protocol, ioe);
}
}
@Override public void requestHeadersStart(Call call) {
super.requestHeadersStart(call);
mHttpData.stepCode = HttpEventStep.requestHeadersStart;
if (subEventListener != null) {
subEventListener.requestHeadersStart(call);
}
}
@Override public void requestHeadersEnd(Call call, Request request) {
super.requestHeadersEnd(call, request);
mHttpData.requestHeaders = request.headers().toString();
if (subEventListener != null) {
subEventListener.requestHeadersEnd(call, request);
}
}
@Override public void responseHeadersStart(Call call) {
super.responseHeadersStart(call);
mHttpData.stepCode = HttpEventStep.responseHeadersStart;
if (subEventListener != null) {
subEventListener.responseHeadersStart(call);
}
}
@Override public void responseHeadersEnd(Call call, Response response) {
super.responseHeadersEnd(call, response);
if (subEventListener != null) {
subEventListener.responseHeadersEnd(call, response);
}
mHttpData.responseCode = response.code();
if (response.headers() != null) {
mQttpData.responseHeaders = response.headers().toString();
}
if (mHttpData.responseCode != 200) {
try {
String errorJsonString = getErrorJsonString(response);
mHttpData.errorMsg = errorJsonString;
mHttpData.errorCode = getErrorCode(errorJsonString);
} catch (Exception e) {
e.printStackTrace();
}
}
}
@Override public void requestBodyStart(Call call) {
super.requestBodyStart(call);
mHttpData.stepCode = HttpEventStep.requestBodyStart;
if (subEventListener != null) {
subEventListener.requestBodyStart(call);
}
}
@Override public void requestBodyEnd(Call call, long byteCount) {
super.requestBodyEnd(call, byteCount);
mHttpData.requestByteCount = byteCount;
if (subEventListener != null) {
subEventListener.requestBodyEnd(call, byteCount);
}
}
@Override public void responseBodyStart(Call call) {
super.responseBodyStart(call);
mHttpData.stepCode = HttpEventStep.responseBodyStart;
if (subEventListener != null) {
subEventListener.responseBodyStart(call);
}
}
@Override public void responseBodyEnd(Call call, long byteCount) {
super.responseBodyEnd(call, byteCount);
mHttpData.responseByteCount = byteCount;
if (subEventListener != null) {
subEventListener.requestBodyEnd(call, byteCount);
}
}
@Override public void callStart(Call call) {
super.callStart(call);
mHttpData.stepCode = HttpEventStep.callStart;
log("callStart");
if (subEventListener != null) {
subEventListener.callStart(call);
}
}
@Override public void callFailed(Call call, IOException ioe) {
super.callFailed(call, ioe);
log("callFailed");
if (subEventListener != null) {
subEventListener.callFailed(call, ioe);
}
if (callStartNanos <= 0) {
return;
}
long totalCost = getCost(callStartNanos);
if (totalCost <= 0) {
return;
}
mHttpData.updateByCall(call);
if (SuMonitorBlackList.inBlackList(mHttpData.url)) {
//黑名单不埋点
return;
}
if (!NetworkUtil.isNetworkConnected(SuHttpMonitor.getContext())) {
return;
}
mHttpData.totalCost = totalCost;
mHttpData.errorMsg = "callFailed";
if (ioe != null) {
StringBuilder stringBuilder = new StringBuilder();
stringBuilder.append(mHttpData.stepCode.name())
.append(",")
.append(NetworkUtil.getNetWorkMsg())
.append(",Ex:")
.append(ioe.getClass().getSimpleName())
.append(",Msg:")
.append(ioe.getMessage())
.append(",trace:");
StackTraceElement[] stackTraceElements = ioe.getStackTrace();
if (stackTraceElements != null && stackTraceElements.length > 0) {
stringBuilder.append(stackTraceElements[0].toString());
}
mHttpData.errorMsg = stringBuilder.toString();
}
}
@Override public void callEnd(Call call) {
super.callEnd(call);
log("callEnd");
if (subEventListener != null) {
subEventListener.callEnd(call);
}
mHttpData.updateByCall(call);
if (SuMonitorBlackList.inBlackList(mHttpData.url)) {
return;
}
if (callStartNanos <= 0) {
return;
}
long totalCost = getCost(callStartNanos);
if (totalCost <= 0) {
return;
}
mHttpData.methodName = call.request().url().encodedPath();
mHttpData.method = call.request().method();
mHttpData.totalCost = totalCost;
try {
mHttpData.requestParams = getRequestParams(call.request());
} catch (Exception e) {
e.printStackTrace();
}
}
private long getCost(long startNano) {
return TimeUnit.NANOSECONDS.toMillis(System.nanoTime() - startNano);
}
private int getErrorCode(String errorJsonStr) {
if (!TextUtils.isEmpty(errorJsonStr)) {
JSONObject errorJson;
try {
errorJson = new JSONObject(errorJsonStr);
return errorJson.optInt("errorCode");
} catch (JSONException e) {
e.printStackTrace();
}
}
return 0;
}
private String getErrorJsonString(Response response) throws Exception {
ResponseBody responseBody = response.body();
if (responseBody != null && response.code() != 200) {
BufferedSource source = responseBody.source();
// Buffer the entire body.
try {
source.request(Long.MAX_VALUE);
} catch (IOException e) {
e.printStackTrace();
}
Buffer buffer = source.buffer();
Charset charset = UTF8;
MediaType contentType = responseBody.contentType();
if (contentType != null) {
charset = contentType.charset(UTF8);
}
if (isPlaintext(buffer) && charset != null) {
return new String(buffer.clone().readByteArray(), charset);
}
}
return null;
}
private static boolean isPlaintext(Buffer buffer) {
try {
Buffer prefix = new Buffer();
long byteCount = buffer.size() < 64 ? buffer.size() : 64;
buffer.copyTo(prefix, 0, byteCount);
for (int i = 0; i < 16; i++) {
if (prefix.exhausted()) {
break;
}
int codePoint = prefix.readUtf8CodePoint();
if (Character.isISOControl(codePoint) && !Character.isWhitespace(codePoint)) {
return false;
}
}
return true;
} catch (EOFException e) {
// Truncated UTF-8 sequence.
return false;
}
}
private static String getRequestParams(Request request) throws Exception {
RequestBody requestBody = request.body();
boolean hasRequestBody = requestBody != null;
if (!hasRequestBody) {
return null;
}
Buffer buffer = new Buffer();
requestBody.writeTo(buffer);
Charset charset = UTF8;
MediaType contentType = requestBody.contentType();
if (contentType != null) {
charset = contentType.charset(UTF8);
}
String param = null;
if (isPlaintext(buffer) && charset != null) {
param = URLDecoder.decode(new String(buffer.readByteArray(), charset));
}
return param;
}
private void log(String message) {
}
}
public final class CallbackBuilder {
public EventListener.Factory factory;
public SuKVEventListener stepCallback;
public boolean isDebug;
}
public interface SuKVEventListener {
//通知业务测上报
void onKVEvent(String eventId, HashMap<String, String> params);
}
class HttpData {
/**
* {{@link Proxy}}
*/
public String proxy;
/**
* {{@link InetSocketAddress}}
*/
public String inetSocketAddress;
/**
* {{@link Protocol}}
*/
public String protocol;
// http or https
public String schema;
// 域名 medi.rthdo.com
public String domain;
// GET,POST
public String method;
// 服务名 api/rest/xxx/xxx
public String methodName;
// url
public String url;
//DNS的耗时
public Long dnsCost;
//建连耗时
public Long connectCost;
//服务器响应耗时
public long responseCost;
//网络请求总耗时
public long totalCost;
//TraceId,用于全链路监控
public String traceId;
//请求参数
public String requestParams;
//错误信息
public String errorMsg;
//本地错误码-40,-20
public HttpEventStep stepCode = HttpEventStep.begin;
//Http状态码 200 ,404
public Integer responseCode;
//小影错误码,无用
public int errorCode;
//请求包体大小
public long requestByteCount;
//响应包体大小
public long responseByteCount;
//请求头
public String requestHeaders;
//响应头
public String responseHeaders;
public void updateByCall(Call call) {
schema = call.request().url().scheme();
domain = call.request().url().host();
method = call.request().method();
methodName = call.request().url().encodedPath();
url = call.request().url().toString();
}
public String getStatusCode() {
int statusCode;
if (responseCode != null && responseCode != 0) {
statusCode = responseCode;
} else {
statusCode = stepCode.getClientStatusCode();
}
return String.valueOf(statusCode);
}
public boolean isFirst() {
return dns() && connect();
}
public boolean dns() {
return dnsCost != null && dnsCost >= 0;
}
public boolean connect() {
return connectCost != null
&& connectCost >= 0;
}
}