Flink源码分析之累加器监控

5 篇文章 0 订阅
2 篇文章 0 订阅

我们再探索累加器监控如何获取,跟着截图一探到底吧:
1.
在这里插入图片描述2.然后我们找到node.js渲染模块
index.js 文件
在这里插入图片描述找到这里就知道flinkweb前端做数据处理写的好复杂样子,是那些开发者故意写这么复杂让我们看不懂吗?

3.根据类似ajax请求知道请求了job的控制器获取信息的。来我们看下后端模块如何坑的:
JobAccumulatorsInfo.java

在这里插入图片描述4.我们搞懂这个类外界如何调用的呢,于是我们找啊找找到JobAccumulatorsHandler.java类

/*
 * Licensed to the Apache Software Foundation (ASF) under one
 * or more contributor license agreements.  See the NOTICE file
 * distributed with this work for additional information
 * regarding copyright ownership.  The ASF licenses this file
 * to you under the Apache License, Version 2.0 (the
 * "License"); you may not use this file except in compliance
 * with the License.  You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package org.apache.flink.runtime.rest.handler.job;

import org.apache.flink.api.common.time.Time;
import org.apache.flink.runtime.accumulators.StringifiedAccumulatorResult;
import org.apache.flink.runtime.executiongraph.AccessExecutionGraph;
import org.apache.flink.runtime.rest.handler.HandlerRequest;
import org.apache.flink.runtime.rest.handler.RestHandlerException;
import org.apache.flink.runtime.rest.handler.legacy.ExecutionGraphCache;
import org.apache.flink.runtime.rest.messages.AccumulatorsIncludeSerializedValueQueryParameter;
import org.apache.flink.runtime.rest.messages.EmptyRequestBody;
import org.apache.flink.runtime.rest.messages.JobAccumulatorsInfo;
import org.apache.flink.runtime.rest.messages.JobAccumulatorsMessageParameters;
import org.apache.flink.runtime.rest.messages.JobIDPathParameter;
import org.apache.flink.runtime.rest.messages.MessageHeaders;
import org.apache.flink.runtime.rest.messages.ResponseBody;
import org.apache.flink.runtime.webmonitor.RestfulGateway;
import org.apache.flink.runtime.webmonitor.history.ArchivedJson;
import org.apache.flink.runtime.webmonitor.history.JsonArchivist;
import org.apache.flink.runtime.webmonitor.retriever.GatewayRetriever;
import org.apache.flink.util.OptionalFailure;
import org.apache.flink.util.SerializedValue;

import java.io.IOException;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.Executor;

/**
 * Request handler that returns the aggregated accumulators of a job.
 */
public class JobAccumulatorsHandler extends AbstractExecutionGraphHandler<JobAccumulatorsInfo, JobAccumulatorsMessageParameters> implements JsonArchivist {

	public JobAccumulatorsHandler(
			CompletableFuture<String> localRestAddress,
			GatewayRetriever<? extends RestfulGateway> leaderRetriever,
			Time timeout,
			Map<String, String> responseHeaders,
			MessageHeaders<EmptyRequestBody, JobAccumulatorsInfo, JobAccumulatorsMessageParameters> messageHeaders,
			ExecutionGraphCache executionGraphCache,
			Executor executor) {
		super(
			localRestAddress,
			leaderRetriever,
			timeout,
			responseHeaders,
			messageHeaders,
			executionGraphCache,
			executor);
	}

	@Override
	protected JobAccumulatorsInfo handleRequest(HandlerRequest<EmptyRequestBody, JobAccumulatorsMessageParameters> request, AccessExecutionGraph graph) throws RestHandlerException {
		List<Boolean> queryParams = request.getQueryParameter(AccumulatorsIncludeSerializedValueQueryParameter.class);

		final boolean includeSerializedValue;
		if (!queryParams.isEmpty()) {
			includeSerializedValue = queryParams.get(0);
		} else {
			includeSerializedValue = false;
		}

		return createJobAccumulatorsInfo(graph, includeSerializedValue);
	}

	@Override
	public Collection<ArchivedJson> archiveJsonWithPath(AccessExecutionGraph graph) throws IOException {
		ResponseBody json = createJobAccumulatorsInfo(graph, true);
		String path = getMessageHeaders().getTargetRestEndpointURL()
			.replace(':' + JobIDPathParameter.KEY, graph.getJobID().toString());
		return Collections.singleton(new ArchivedJson(path, json));
	}

	private static JobAccumulatorsInfo createJobAccumulatorsInfo(AccessExecutionGraph graph, boolean includeSerializedValue) {
		StringifiedAccumulatorResult[] stringifiedAccs = graph.getAccumulatorResultsStringified();
		List<JobAccumulatorsInfo.UserTaskAccumulator> userTaskAccumulators = new ArrayList<>(stringifiedAccs.length);

		for (StringifiedAccumulatorResult acc : stringifiedAccs) {
			userTaskAccumulators.add(
				new JobAccumulatorsInfo.UserTaskAccumulator(
					acc.getName(),
					acc.getType(),
					acc.getValue()));
		}

		JobAccumulatorsInfo accumulatorsInfo;
		if (includeSerializedValue) {
			Map<String, SerializedValue<OptionalFailure<Object>>> serializedUserTaskAccumulators = graph.getAccumulatorsSerialized();
			accumulatorsInfo = new JobAccumulatorsInfo(Collections.emptyList(), userTaskAccumulators, serializedUserTaskAccumulators);
		} else {
			accumulatorsInfo = new JobAccumulatorsInfo(Collections.emptyList(), userTaskAccumulators, Collections.emptyMap());
		}

		return accumulatorsInfo;
	}
}

5.这里就明白后端和前端写的真漂亮,老复杂了,新手看web源码时候要注意前端路由的坑。
在这里插入图片描述

  • 2
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 2
    评论
CMS LITE(雄迈cms监控软件)是雄迈推出的一款远程视频监控客户端,cms监控软件配套摄像机使用方便快捷,用户可以在计算机上就可以了解监控情况,实时对周边环境进行监控,有需要的赶快下载吧! CMS LITE(雄迈cms监控软件)基本简介 杭州雄迈信息技术有限公司是业内最具发展潜力的优质企业,以领先的监控产品创新技术为核心竞争力,致力于提供高品质、高性价比的安防产品,提供客户化的产品和点到点的服务。 公司的主要产品有硬盘录像机、网络视频服务器、便携式硬盘录像机、网络摄像机、物联网前后端产品、云监控平台等,涵盖道路监控、智能交通、城市安防、车载监控、小区监控、物联网等众多领域,被广泛应用于金融、交通、电信、电力、水利等行业。 企业专注于数字音视频压缩、存储、解压缩及网络传输等相关领域的深入研究,同时不断挑战新技术新方法,独创BLOCK结构,解决了行业内细分领域差异化产品的难题,使快速灵活定制成为可能。2012年5月公司重拳推出集高清、智能、混合于一体的新型硬盘录像机,结合领先的视频云监控系统,真正实现了即插即用,一步上网功能,引领家用安防的普及化,率先步入DVR4.0云监控时代,掀起安防监控新篇章。 公司产品畅销海内外市场,营销及服务网络遍及全球,目前在国内各大城市已设立分公司和办事处,主要出口国家有美国、英国、法国、德国、巴西、印度、俄罗斯、意大利、波兰等,并在多个国家建立了服务器集群,让世界各地的客户体验云服务带来的快捷和便利。
评论 2
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值