服务追踪调研

OpenTracing

简介

OpenTracing 提供了一套平台无关、厂商无关的 API,这样不同的组织或者开发人员就能够更加方便的添加或更换追踪系统的实现。 OpenTracing API 中的一些概念和术语,在不同的语言环境下都是共享。

接口规范

  • Tracer Interface
  • Span Interface
  • SpanContext Interface
  • Global and No-op Tracers

jaeger install

# run jaeger
docker run -d \
  --rm \
  --name jaeger \
  -p6831:6831/udp \
  -p16686:16686 \
  jaegertracing/all-in-one:1.16.0
  
# run demo  
docker run -d \
  --name hotrod\
  --rm \
  --link jaeger \
  --env JAEGER_AGENT_HOST=jaeger \
  --env JAEGER_AGENT_PORT=6831 \
  -p8080-8083:8080-8083 \
  jaegertracing/example-hotrod:1.16.0 \
  all

jaeger 组件简介

jaeger-agent

jaeger client sdk 产生的数据由 jaeger-agent负责采集并发送到jaeger-collector

Port Protocol Function
6831 UDP accept jaeger.thrift in compact Thrift protocol used by most current Jaeger clients
6832 UDP accept jaeger.thrift in binary Thrift protocol used by Node.js Jaeger client (because thriftrw npm package does not support compact protocol)
5778 HTTP serve configs, sampling strategies
5775 UDP accept zipkin.thrift in compact Thrift protocol (deprecated; only used by very old Jaeger clients, circa 2016)
14271 HTTP Healthcheck at / and metrics at /metrics

jaeger-collector

jaeger-collector 收集 agent发送过来的数据并分发给后端存储

Port Protocol Function
14267 TChanel used by jaeger-agent to send spans in jaeger.thrift format
14250 gRPC used by jaeger-agent to send spans in model.proto format
14268 HTTP can accept spans directly from clients in jaeger.thrift format over binary thrift protocol
9411 HTTP can accept Zipkin spans in Thrift, JSON and Proto (disabled by default)
14269 HTTP Healthcheck at / and metrics at /metrics

Storage Backends

通过环境变量 SPAN_STORAGE_TYPE 来设置后端存储

  • badger和Memory 仅在all-in-one安装模式下使用
  • elasticsearch
  • cassandra noSql数据库
  • kafka 仅作为缓存,将数据分发到es或者Cassandra

Storage plugin

jaeger 支持 gRPC通信的后端存储

jaeger-ingester

jaeger-ingester可以消费kafka内的数据分发到es或者cassandra

Port Protocol Function
14270 HTTP Healthcheck at / and metrics at /metrics

jaeger-query

该服务是无状态应用,一般部署在负载均衡(nginx)后面

Port Protocol Function
16686 HTTP /api/* endpoints and Jaeger UI at /
16687 HTTP Healthcheck at / and metrics at /metrics

spark-dependencies

生产环境下需要部署spark-dependencies 分析调用链信息

验证Jaeger Client自动埋点

java-specialagent

java-specialagent是java应用自动埋点第三方引擎库,详细说明请点击github传送门

  • 目前测试情况是springboot web应用和采用okhttp和Dropwizard框架编译的java应用都可以实现自动埋点,但是阿里的sofarpc无法自动埋点。

  • 添加自动埋点的java应用会延长启动时间

本地运行 jaeger all-in-one应用

docker run -d --rm \
    --name jaeger \
    -e COLLECTOR_ZIPKIN_HTTP_PORT=9411 \
    -p 5775:5775/udp \
    -p 6831:6831/udp \
    -p 6832:6832/udp \
    -p 5778:5778 \
    -p 16686:16686 \
    -p 14268:14268 \
    -p 14250:14250 \
    -p 9411:9411 \
    jaegertracing/all-in-one:1.16.0

运行 springboot demo

验证可行,能够在jaeger ui中查询到服务追踪信息

java -javaagent:opentracing-specialagent-1.5.8.jar -Dsa.tracer=jaeger -Dsa.log.level=INFO  -DJAEGER_SERVICE_NAME="springboot" -jar springboot-helloworld-1.0-SNAPSHOT.jar --server.port=9000

运行PRC demo

验证可行,能够在jaeger ui中查询到服务追踪信息

java -javaagent:opentracing-specialagent-1.5.8.jar -Dsa.tracer=jaeger -Dsa.log.level=INFO  -DJAEGER_SERVICE_NAME="formatter" -jar jaeger-demo/formatter/target/formatter-1.0-SNAPSHOT.jar

java -javaagent:opentracing-specialagent-1.5.8.jar -Dsa.tracer=jaeger -Dsa.log.level=INFO  -DJAEGER_SERVICE_NAME="publisher" -jar jaeger-demo/publisher/target/publisher-1.0-SNAPSHOT.jar

java -javaagent:opentracing-specialagent-1.5.8.jar -Dsa.tracer=jaeger -Dsa.log.level=INFO  -DJAEGER_SERVICE_NAME="callserver" -jar jaeger-demo/callserver/target/callserver-1.0-SNAPSHOT.jar

运行sofarpc demo

验证失败,在jaeger ui中没有查询到追踪信息

  • rpcclient 第一次发起请求会报错,后续请求可正常收到响应
  • 报错信息如下
2020-03-03 14:45:50.730  INFO 34856 --- [           main] com.citicbank.RpcclientApplication       : Started RpcclientApplication in 7.443 seconds (JVM running for 19.898)
com.alipay.sofa.rpc.context.RpcInvokeContext@5013c6bc{timeout=null, targetURL='null', targetGroup='null', responseCallback=null, future=null, map={}, requestBaggage={service=mosn}, responseBaggage={}}
Exception in thread "main" java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:50)
	at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51)
Caused by: com.alipay.sofa.rpc.core.exception.SofaTimeOutException: com.alipay.remoting.rpc.exception.InvokeTimeoutException: Rpc invocation timeout[responseCommand TIMEOUT]! the address is 127.0.0.1:12200
	at com.alipay.sofa.rpc.transport.bolt.BoltClientTransport.convertToRpcException(BoltClientTransport.java:327)
	at com.alipay.sofa.rpc.transport.bolt.BoltClientTransport.syncSend(BoltClientTransport.java:251)
	at com.alipay.sofa.rpc.client.AbstractCluster.doSendMsg(AbstractCluster.java:535)
	at com.alipay.sofa.rpc.client.AbstractCluster.sendMsg(AbstractCluster.java:506)
	at com.alipay.sofa.rpc.filter.ConsumerInvoker.invoke(ConsumerInvoker.java:60)
	at com.alipay.sofa.rpc.filter.sofatracer.ConsumerTracerFilter.invoke(ConsumerTracerFilter.java:66)
	at com.alipay.sofa.rpc.filter.FilterInvoker.invoke(FilterInvoker.java:98)
	at com.alipay.sofa.rpc.filter.RpcReferenceContextFilter.invoke(RpcReferenceContextFilter.java:80)
	at com.alipay.sofa.rpc.filter.FilterInvoker.invoke(FilterInvoker.java:98)
	at com.alipay.sofa.rpc.filter.ConsumerExceptionFilter.invoke(ConsumerExceptionFilter.java:37)
	at com.alipay.sofa.rpc.filter.FilterInvoker.invoke(FilterInvoker.java:98)
	at com.alipay.sofa.rpc.filter.FilterChain.invoke(FilterChain.java:262)
	at com.alipay.sofa.rpc.client.AbstractCluster.filterChain(AbstractCluster.java:499)
	at com.alipay.sofa.rpc.client.FailoverCluster.doInvoke(FailoverCluster.java:67)
	at com.alipay.sofa.rpc.client.AbstractCluster.invoke(AbstractCluster.java:288)
	at com.alipay.sofa.rpc.client.ClientProxyInvoker.invoke(ClientProxyInvoker.java:83)
	at com.citicbank.service.RpcDemoService_proxy_0.sayHello(RpcDemoService_proxy_0.java)
	at com.citicbank.service.RpcDemoClientImpl.sayClientAnnotation(RpcDemoClientImpl.java:19)
	at com.citicbank.RpcclientApplication.main(RpcclientApplication.java:22)
	... 8 more
Caused by: com.alipay.remoting.rpc.exception.InvokeTimeoutException: Rpc invocation timeout[responseCommand TIMEOUT]! the address is 127.0.0.1:12200
	at com.alipay.remoting.rpc.RpcResponseResolver.preProcess(RpcResponseResolver.java:83)
	at com.alipay.remoting.rpc.RpcResponseResolver.resolveResponseObject(RpcResponseResolver.java:54)
	at com.alipay.remoting.rpc.RpcRemoting.invokeSync(RpcRemoting.java:186)
	at com.alipay.remoting.rpc.RpcClientRemoting.invokeSync(RpcClientRemoting.java:64)
	at com.alipay.remoting.rpc.RpcClient.invokeSync(RpcClient.java:355)
	at com.alipay.sofa.rpc.transport.bolt.BoltClientTransport.doInvokeSync(BoltClientTransport.java:275)
	at com.alipay.sofa.rpc.transport.bolt.BoltClientTransport.syncSend(BoltClientTransport.java:248)
	... 25 more
Unknown instrumentable state for thread: DestroyJavaVM

运行命令

java -javaagent:opentracing-specialagent-1.5.8.jar -Dsa.tracer=jaeger -Dsa.log.level=INFO  -DJAEGER_SERVICE_NAME="client" -jar rpcclient-0.0.1-SNAPSHOT.jar

java -javaagent:opentracing-specialagent-1.5.8.jar -Dsa.tracer=jaeger -Dsa.log.level=INFO  -DJAEGER_SERVICE_NAME="server" -jar rpcserver-0.0.1-SNAPSHOT.jar

java-specialagent 支持的插件

OpenTracing Plugin
(link to plugin implementation)
SpecialAgent Rule
(link to rule implementation)
Min Version
(min supported)
Max Version
(max supported)
Akka Actor [akka:actor][akka-actor] 2.5.0 2.6.1
Akka Http [akka:http][akka-http] 10.1.0 10.1.10
Apache Camel [camel][camel] 2.24.0 2.24.2
Apache HttpClient [apache:httpclient][apache-httpclient] 4.4 4.5.9
Async Http Client [asynchttpclient][asynchttpclient] 2.7.0 2.10.4
AWS SDK [aws:sdk:1][aws-sdk-1] 1.11.79 1.11.5.1
  [aws:sdk:2][aws-sdk-2] 2.1.4 2.7.15
Cassandra Driver [cassandra:driver:3][cassandra-driver-3] 3.0.0 3.7.2
  [cassandra:driver:4][cassandra-driver-4] 4.0.0 4.2.0
Couchbase Client [couchbase-client][couchbase-client] 2.7.3 2.7.11
Dynamic
 
[dynamic
(configurable)][dynamic]
*
 
*
 
Elasticsearch Client
 
[elasticsearch:client-transport][elasticsearch-7-transport-client]
[elasticsearch:client-rest][elasticsearch-7-rest-client]
6.4.0
 
7.3.1
 
Feign [feign][feign] 9.0.0 10.4.0
Google Http Client [google-http-client][google-http-client] 1.19.0 1.33.0
Grizzly AsyncHttpClient [grizzly:ahc][grizzly-ahc] 1.15 1.15
Grizzly HTTP Server [grizzly:http-server][grizzly-http-server] 2.3.35 2.3.35
GRPC [grpc][grpc] 1.6.0 1.23.0
Hazelcast [hazelcast][hazelcast] 3.12 3.12.4
Java Concurrent API [java.util.concurrent] [concurrent][concurrent] 1.5 11
[Java JDBC API [java.sql]][java-jdbc]
 
[jdbc
(configurable)][jdbc]
3.1
 
4.3
 
[Java JMS API [javax.jms]][java-jms] [jms:1][jms-1] 1.1 1.1
  [jms:2][jms-2] 2.0 2.0a
Java Servlet API [javax.servlet]
 
[servlet
(configurable)][servlet]
2.3
 
3.1
 
    Jetty   7.6.21.v20160908 9.2.15.v20160210
    Tomcat   7.0.65 9.0.27
Java Thread [java.lang.Thread] [thread][thread] 1.0 11
HttpURLConnection [java.net.HttpURLConnection] [httpurlconnection][httpurlconnection] 1.1 11
JAX-RS Client [jax-rs][jaxrs] 2.0 2.1
Jedis Client [jedis][jedis] 2.7.0 3.1.0
Kafka Client [kafka:client][kafka-client] 1.1.0 2.3.0
Lettuce Client [lettuce:5.0][lettuce-5.0] 5.0.0.RELEASE 5.0.5.RELEASE
  [lettuce:5.1][lettuce-5.1] 5.1.0.M1 5.1.8.RELEASE
  [lettuce:5.2][lettuce-5.2] 5.2.0.RELEASE 5.2.0.RELEASE
MongoDB Driver [mongo:driver][mongo-driver] 3.9.0 3.11.0
Neo4j Driver [neo4j:driver][neo4j-driver] 4.0.0 4.0.0
Netty [netty][netty] 4.1.0 4.1.44
[OkHttp][java-okhttp] [okhttp][okhttp] 3.5.0 4.2.2
Play Framework [play][play] 2.6.0 2.7.3
Play WS [play:ws][play-ws] 2.0.0 2.0.7
Pulsar Client [pulsar:client][pulsar-client] 2.2.0 2.4.2
Pulsar Functions 
 
[pulsar-functions
(configurable)][pulsar-functions]
2.2.0
 
2.4.2
 
RabbitMQ Client [rabbitmq:client][rabbitmq-client] 5.0.0 5.7.3
Reactor [reactor][reactor] 3.2.3.RELEASE 3.2.3.RELEASE
Redisson [redisson][redisson] 3.11.0 3.11.5
RxJava 2 [rxjava:2][rxjava-2] 2.1.0 2.2.12
Spring JMS [spring:jms][spring-jms] 5.0.0.RELEASE 5.2.2.RELEASE
Spring Kafka [spring:kafka][spring-kafka] 2.2.0.RELEASE 2.3.3.RELEASE
Spring Messaging [spring:messaging][spring-messaging] 5.1.0.RELEASE 5.2.2.RELEASE
Spring RabbitMQ [spring:rabbitmq][spring-rabbitmq] 2.0.0.RELEASE 2.2.1.RELEASE
Spring WebFlux [spring:webflux][spring-webflux] 5.1.0.RELEASE 5.2.1.RELEASE
Spring Boot WebSocket STOMP [spring:websocket][spring-websocket] 2.1.0.RELEASE 2.1.8.RELEASE
Spring [@Async and @Scheduled] [spring:scheduling][spring-scheduling] 5.0.0.RELEASE 5.2.1.RELEASE
Spring Web [spring:web:3][spring-web-3] 3.0.3.RELEASE 3.2.18.RELEASE
  [spring:web:4.0][spring-web-4.0] 4.0.0.RELEASE 4.0.9.RELEASE
  [spring:web:4.x][spring-web-4] 4.1.0.RELEASE 4.3.25.RELEASE
  [spring:web:5][spring-web-5] 5.0.0.RELEASE 5.2.1.RELEASE
Spring Web MVC [spring:webmvc:3][spring-webmvc-3] 3.0.2.RELEASE 3.2.18.RELEASE
  [spring:webmvc:4][spring-webmvc-4] 4.0.0.RELEASE 4.3.25.RELEASE
  [spring:webmvc:5][spring-webmvc-5] 5.0.0.RELEASE 5.2.1.RELEASE
Spymemcached [spymemcached][spymemcached] 2.11.0 2.12.3
Thrift [thrift][thrift] 0.10.0 0.12.0
Zuul [zuul][zuul] 1.0.0 1.3.1

关于skywalking

Skywalking 是一个APM系统,即应用性能监控系统,为微服务架构和云原生架构系统设计。完全支持opentracing接口,它通过探针自动收集所需的指标,并进行分布式追踪。通过这些调用链路以及指标,Skywalking APM会感知应用间关系和服务间关系,并进行相应的指标统计。目前支持链路追踪和监控应用组件如下,基本涵盖主流框架和容器,包括sofarpc、 dubbo等。详细列表如下

skywalking demo 实例

[北京服务器] http://122.112.182.72:8080/

发布了7 篇原创文章 · 获赞 0 · 访问量 1732
展开阅读全文

没有更多推荐了,返回首页

©️2019 CSDN 皮肤主题: 大白 设计师: CSDN官方博客

分享到微信朋友圈

×

扫一扫,手机浏览