Hystrix Dashboard监控
前言
本篇博文,从源码层次,分析Hystrix Dashboard如何进行Hystrix监控,进而加深Hystrix Dashboard理解。
一、开启Hystrix Dashboard
1、添加依赖
项目中,添加如下依赖。由于项目中,集成了swagger-ui,且swagger-ui使用了高版本的guava,与hystrix所依赖guava冲突,需要移除hystrix所使用的的低版本guava。
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
<version>2.1.3.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-feign</artifactId>
<version>1.4.5.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-hystrix</artifactId>
<version>2.1.3.RELEASE</version>
<exclusions>
<exclusion>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
</exclusion>
</exclusions>
</dependency>
<!-- Hystrix Dashboard监控,所需配置信息 -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
<version>2.1.3.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-hystrix-dashboard</artifactId>
<version>1.4.5.RELEASE</version>
</dependency>
<!-- Hystrix Dashboard监控,所需配置信息 -->
2、配置项
配置文件中,添加如下配置项
management.endpoints.web.exposure.include=hystrix.stream,turbine.stream
3、配置类
在配置类或启动类上,通过EnableHystrixDashboard注解,开启Hystrix Dashboard监控。
@Configuration
@EnableCircuitBreaker
@EnableHystrixDashboard
@EnableTurbine
public class FeignCloudConfiguration {
}
4、成品展示
启动项目,在浏览器中,输入http://localhost:10001/hystrix。展示如下界面
在中间地址栏,输入http://localhost:10001/actuator/hystrix.stream。
在刚开始的时候,页面一直处理loading状态,这个是因为,没有相关服务被调用的原因。当任意调用一个微服务时,此时能正常显示Hystrix监控数据。
二、Hystrix Dashboard源码分析
项目启动时,控制台,存在如下内容
2021-05-12 20:45:40.813 INFO 10180 — [ main] c.netflix.config.DynamicPropertyFactory : DynamicPropertyFactory is initialized with configuration sources: com.netflix.config.ConcurrentCompositeConfiguration@21bd128b
2021-05-12 20:45:43.970 INFO 10180 — [ main] o.s.b.a.e.web.ServletEndpointRegistrar : Registered ‘/actuator/hystrix.stream’ to hystrix.stream-actuator-endpoint
2021-05-12 20:45:44.746 INFO 10180 — [ main] o.s.b.a.e.web.EndpointLinksResolver : Exposing 1 endpoint(s) beneath base path ‘/actuator’
也就是,/actuator/hystrix.stream请求路径,被映射到hystrix.stream-actuator-endpoint端点。该端点,通过ServletEndpointRegistrar完成注册操作。注册动作,对应的源码如下所示
@Override
public void onStartup(ServletContext servletContext) throws ServletException {
this.servletEndpoints
.forEach((servletEndpoint) -> register(servletContext, servletEndpoint));
}
private void register(ServletContext servletContext,
ExposableServletEndpoint endpoint) {
String name = endpoint.getEndpointId().toLowerCaseString() + "-actuator-endpoint";
String path = this.basePath + "/" + endpoint.getRootPath();
String urlMapping = path.endsWith("/") ? path + "*" : path + "/*";
EndpointServlet endpointServlet = endpoint.getEndpointServlet();
Dynamic registration = servletContext.addServlet(name,
endpointServlet.getServlet());
registration.addMapping(urlMapping);
registration.setInitParameters(endpointServlet.getInitParameters());
logger.info("Registered '" + path + "' to " + name);
}
1、ServletEndpointRegistrar装配
ServletEndpointManagementContextConfiguration配置类,存在WebMvcServletEndpointManagementContextConfiguration内部类,正是这个内部类完成ServletEndpointRegistrar 的装配。
@Configuration
@ConditionalOnClass(DispatcherServlet.class)
public static class WebMvcServletEndpointManagementContextConfiguration {
private final ApplicationContext context;
public WebMvcServletEndpointManagementContextConfiguration(ApplicationContext context) {
this.context = context;
}
@Bean
public ServletEndpointRegistrar servletEndpointRegistrar(WebEndpointProperties properties,
ServletEndpointsSupplier servletEndpointsSupplier) {
DispatcherServletPath dispatcherServletPath = this.context.getBean(DispatcherServletPath.class);
return new ServletEndpointRegistrar(dispatcherServletPath.getRelativePath(properties.getBasePath()),
servletEndpointsSupplier.getEndpoints());
}
}
ServletEndpointRegistrar 构造方法中的第一个参数,用于指定请求的basePath,其值对应于WebEndpointProperties.basePath属性,默认为/actuator。
2、端点注册
ServletEndpointRegistrar构造方法的第二个参数,通过ServletEndpointDiscoverer.getEndpoints收集待注册端点信息。
@Override
public final Collection<E> getEndpoints() {
if (this.endpoints == null) {
this.endpoints = discoverEndpoints();
}
return this.endpoints;
}
private Collection<E> discoverEndpoints() {
Collection<EndpointBean> endpointBeans = createEndpointBeans();
addExtensionBeans(endpointBeans);
return convertToEndpoints(endpointBeans);
}
private Collection<EndpointBean> createEndpointBeans() {
Map<EndpointId, EndpointBean> byId = new LinkedHashMap<>();
String[] beanNames = BeanFactoryUtils.beanNamesForAnnotationIncludingAncestors(
this.applicationContext, Endpoint.class);
for (String beanName : beanNames) {
if (!ScopedProxyUtils.isScopedTarget(beanName)) {
EndpointBean endpointBean = createEndpointBean(beanName);
EndpointBean previous = byId.putIfAbsent(endpointBean.getId(),
endpointBean);
Assert.state(previous == null,
() -> "Found two endpoints with the id '" + endpointBean.getId()
+ "': '" + endpointBean.getBeanName() + "' and '"
+ previous.getBeanName() + "'");
}
}
return byId.values();
}
createEndpointBeans方法,首先收集Spring上下文环境中,存在EndPoint注解标注的bean,然后构建成EndpointBean对象。该对象中,有一个比较重要的属性,即filter属性,后面的过滤动作,是通过该filter完成。
private Class<?> getFilter(Class<?> type) {
AnnotationAttributes attributes = AnnotatedElementUtils.getMergedAnnotationAttributes(type, FilteredEndpoint.class);
if (attributes == null) {
return null;
}
return attributes.getClass("value");
}
也就是,通过获取Bean上的FilteredEndpoint注解,拿到该Bean所使用的的filter。
private Collection<E> convertToEndpoints(Collection<EndpointBean> endpointBeans) {
Set<E> endpoints = new LinkedHashSet<>();
for (EndpointBean endpointBean : endpointBeans) {
if (isEndpointExposed(endpointBean)) {
endpoints.add(convertToEndpoint(endpointBean));
}
}
return Collections.unmodifiableSet(endpoints);
}
private boolean isEndpointExposed(EndpointBean endpointBean) {
return isFilterMatch(endpointBean.getFilter(), endpointBean)
&& !isEndpointFiltered(endpointBean)
&& isEndpointExposed(endpointBean.getBean());
}
convertToEndpoints方法,使用EndPoint注解标记的Bean,通过isEndpointExposed方法,进行过滤。通过过滤操作后,收集并返回,用于后期的注册动作。
private boolean isFilterMatch(Class<?> filter, EndpointBean endpointBean) {
if (!isEndpointExposed(endpointBean.getBean())) {
return false;
}
if (filter == null) {
return true;
}
E endpoint = getFilterEndpoint(endpointBean);
Class<?> generic = ResolvableType.forClass(EndpointFilter.class, filter)
.resolveGeneric(0);
if (generic == null || generic.isInstance(endpoint)) {
EndpointFilter<E> instance = (EndpointFilter<E>) BeanUtils
.instantiateClass(filter);
return isFilterMatch(instance, endpoint);
}
return false;
}
isFilterMatch方法,首先通过isEndpointExposed方法,判断endPointBean是否使用ServletEndpoint注解标记(该方法被ServletEndpointDiscoverer实现类覆写),然后实例化bean初始化时,所获取的filter属性,该属性须为EndpointFilter实例。然后,通过其match方法,完成匹配操作。
Endpoint收集完成后,我们在回到register方法,此时通过this.basePath + “/” + id构成请求路径,用于映射Endpoint所使用的Servlet。
private void register(ServletContext servletContext,
ExposableServletEndpoint endpoint) {
String name = endpoint.getEndpointId().toLowerCaseString() + "-actuator-endpoint";
String path = this.basePath + "/" + endpoint.getRootPath();
String urlMapping = path.endsWith("/") ? path + "*" : path + "/*";
EndpointServlet endpointServlet = endpoint.getEndpointServlet();
Dynamic registration = servletContext.addServlet(name,
endpointServlet.getServlet());
registration.addMapping(urlMapping);
registration.setInitParameters(endpointServlet.getInitParameters());
logger.info("Registered '" + path + "' to " + name);
}
3、hystrix.stream端点分析
hystrix.stream所使用的的端点为HystrixStreamEndpoint,其中ServletEndpoint为组合注解,包含Endpoint和FilteredEndpoint两个注解。其中,不难发现hystrix.stream对应HystrixMetricsStreamServlet Servlet。
@ServletEndpoint(id = "hystrix.stream")
public class HystrixStreamEndpoint implements Supplier<EndpointServlet> {
private final Map<String, String> initParameters;
public HystrixStreamEndpoint(Map<String, String> initParameters) {
this.initParameters = initParameters;
}
@Override
public EndpointServlet get() {
return new EndpointServlet(HystrixMetricsStreamServlet.class)
.withInitParameters(this.initParameters);
}
}
4、hystrix.stream请求分析
@Override
protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
if (isDestroyed) {
response.sendError(503, "Service has been shut down.");
} else {
handleRequest(request, response);
}
}
private void handleRequest(HttpServletRequest request, final HttpServletResponse response) throws ServletException, IOException {
final AtomicBoolean moreDataWillBeSent = new AtomicBoolean(true);
Subscription sampleSubscription = null;
/* ensure we aren't allowing more connections than we want */
int numberConnections = incrementAndGetCurrentConcurrentConnections();
try {
int maxNumberConnectionsAllowed = getMaxNumberConcurrentConnectionsAllowed(); //may change at runtime, so look this up for each request
if (numberConnections > maxNumberConnectionsAllowed) {
response.sendError(503, "MaxConcurrentConnections reached: " + maxNumberConnectionsAllowed);
} else {
/* initialize response */
response.setHeader("Content-Type", "text/event-stream;charset=UTF-8");
response.setHeader("Cache-Control", "no-cache, no-store, max-age=0, must-revalidate");
response.setHeader("Pragma", "no-cache");
final PrintWriter writer = response.getWriter();
//since the sample stream is based on Observable.interval, events will get published on an RxComputation thread
//since writing to the servlet response is blocking, use the Rx IO thread for the write that occurs in the onNext
sampleSubscription = sampleStream
.observeOn(Schedulers.io())
.subscribe(new Subscriber<String>() {
@Override
public void onCompleted() {
logger.error("HystrixSampleSseServlet: ({}) received unexpected OnCompleted from sample stream", getClass().getSimpleName());
moreDataWillBeSent.set(false);
}
@Override
public void onError(Throwable e) {
moreDataWillBeSent.set(false);
}
@Override
public void onNext(String sampleDataAsString) {
if (sampleDataAsString != null) {
writer.print("data: " + sampleDataAsString + "\n\n");
// explicitly check for client disconnect - PrintWriter does not throw exceptions
if (writer.checkError()) {
moreDataWillBeSent.set(false);
}
writer.flush();
}
}
});
while (moreDataWillBeSent.get() && !isDestroyed) {
try {
Thread.sleep(pausePollerThreadDelayInMs);
//in case stream has not started emitting yet, catch any clients which connect/disconnect before emits start
writer.print("ping: \n\n");
// explicitly check for client disconnect - PrintWriter does not throw exceptions
if (writer.checkError()) {
moreDataWillBeSent.set(false);
}
writer.flush();
} catch (InterruptedException e) {
moreDataWillBeSent.set(false);
}
}
}
} finally {
decrementCurrentConcurrentConnections();
if (sampleSubscription != null && !sampleSubscription.isUnsubscribed()) {
sampleSubscription.unsubscribe();
}
}
}
get请求时,通过设置Content-Type为text/event-stream,基于数据流方式,接收服务端推送的数据。然后通过Observable,接收待发送监控数据后,将其设置为data部分,推送给前端展示。对应数据内容如下图所示
而整个监控数据的源头,对应于如下代码
this.singleSource = Observable.interval(delayInMs, TimeUnit.MILLISECONDS)
.map(new Func1<Long, DashboardData>() {
@Override
public DashboardData call(Long timestamp) {
return new DashboardData(
HystrixCommandMetrics.getInstances(),
HystrixThreadPoolMetrics.getInstances(),
HystrixCollapserMetrics.getInstances()
);
}
})
.doOnSubscribe(new Action0() {
@Override
public void call() {
isSourceCurrentlySubscribed.set(true);
}
})
.doOnUnsubscribe(new Action0() {
@Override
public void call() {
isSourceCurrentlySubscribed.set(false);
}
})
.share()
.onBackpressureDrop();
关于监控数据具体细节,会在后续博文中,继续介绍。
总结
1、hystrix.stream请求,通过HystrixMetricsStreamServlet完成请求操作。
2、HystrixMetricsStreamServlet通过HystrixStreamEndpoint 完成请求路径的映射与绑定。
3、HystrixStreamEndpoint 由于其所使用的EndPoint注解,被ServletEndpointRegistrar
收集,并注册。