Flink REST API #

最新推荐文章于 2024-01-08 10:10:02 发布

CarloPan

最新推荐文章于 2024-01-08 10:10:02 发布

阅读量1.6k

点赞数 1

分类专栏： flink

原文链接：https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/ops/rest_api/

版权

flink 专栏收录该内容

25 篇文章 0 订阅

订阅专栏

官方文档：

https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/ops/rest_api/

https://ci.apache.org/projects/flink/flink-docs-release-1.12/ops/rest_api.html

/jars

Returns a list of all jars previously uploaded via '/jars/upload'.

/jars/upload

Uploads a jar to the cluster.

/jars/:jarid

Deletes a jar previously uploaded via '/jars/upload'.

/jars/:jarid/plan

Returns the dataflow plan of a job contained in a jar previously uploaded via '/jars/upload'. Program arguments can be passed both via the JSON request (recommended) or query parameters.

/jars/:jarid/run

Submits a job by running a jar previously uploaded via '/jars/upload'. Program arguments can be passed both via the JSON request (recommended) or query parameters.

通过运行以前通过“/jar/上传”上传的jar来提交作业。程序参数可以通过json请求（推荐）或查询参数同时传递。

/jobmanager/config

Returns the cluster configuration.

/jobmanager/logs

Returns the list of log files on the JobManager.

从源码来看 Flink 提交作业并调度执行

参考：https://www.cnblogs.com/mrpei/p/12124950.html

从提交来一步一步分析，本文源码基于Apache社区 1.8-release 版本

REST提交作业流程：

1.集群启动后通过 /jars/upload 向集群提交可执行jar文件

2.通过 /jars/:jarid/run 来启动一个job

Apache Flink v1.10 官方中文文档

https://www.bookstack.cn/read/flink-1.10-zh/2edafbfdf4e0cad2.md

Apache Flink v1.9 Documentation

https://www.bookstack.cn/read/apache-flink-1.9/dfa2de84c37e833c.md

Flink 1.7-SNAPSHOT 中文文档

https://wizardforcel.gitbooks.io/flink-17s-doc/content/125.html

监控REST API 译者：flink.sojb.cn

/jars/:jarid /run

动作： POST

响应代码： 200 OK

通过运行先前通过'/ jars / upload'上传的jar来提交作业。

路径参数：

jarid - 标识jar的字符串值。上传jar时会返回一个路径，其中文件名是ID。此值等同于上载的jar（/ jars）列表中的id字段。

查询参数：

program-args （可选）：字符串值，指定程序或计划的参数。
entry-class（可选）：字符串值，指定入口点类的完全限定名称。覆盖jar文件清单中定义的类。
parallelism （可选）：正整数值，指定作业所需的并行度。
allowNonRestoredState （可选）：布尔值，指定如果保存点包含无法映射回作业的状态，是否应拒绝作业提交。
savepointPath （可选）：字符串值，指定要从中还原作业的保存点的路径。

请求：

            {
  "type" : "object",
  "id" : "urn:jsonschema:org:apache:flink:runtime:webmonitor:handlers:JarRunRequestBody",
  "properties" : {
    "entryClass" : {
      "type" : "string"
    },
    "programArgs" : {
      "type" : "string"
    },
    "parallelism" : {
      "type" : "integer"
    },
    "allowNonRestoredState" : {
      "type" : "boolean"
    },
    "savepointPath" : {
      "type" : "string"
    }
  }
}

响应：

            {
  "type" : "object",
  "id" : "urn:jsonschema:org:apache:flink:runtime:webmonitor:handlers:JarRunResponseBody",
  "properties" : {
    "jobid" : {
      "type" : "any"
    }
  }
}

Flink DataStream RestAPI使用

参考：https://blog.csdn.net/wangpei1949/article/details/100853748

Flink使用RestApi

https://blog.csdn.net/YouLoveItY/article/details/107807174

常用的Flink Rest API接口使用

https://blog.csdn.net/weixin_41062002/article/details/106398295?utm_medium=distribute.pc_relevant.none-task-blog-2%7Edefault%7EBlogCommendFromBaidu%7Edefault-4.control&depth_1-utm_source=distribute.pc_relevant.none-task-blog-2%7Edefault%7EBlogCommendFromBaidu%7Edefault-4.control

Flink使用RestApi

flink是一个非常好用的流任务计算框架, 这次我们来试用flink的restApi来提交任务. 主要阐述几个常用的restapi, 包括上传jar包, 查询jar包, 提交任务, 查询任务, 删除任务等,
其它的比如删除jar包, 查询jobmanager, 查询taskmanager等等, 类推就可以得出了, 不在这里进行重复介绍了

1, 上传jar包

   public static boolean uploadJar( File jarFile) {
        RequestBody requestBody = new MultipartBody.Builder()
                .setType(MultipartBody.FORM)
                .addFormDataPart("file", jarFile.getName(),
                    RequestBody.create(MediaType.parse("multipart/form-data"), jarFile))
                .build();

        Request request = new Request.Builder()
                .url("http://host:port/jars/upload")
                .addHeader(userAgent, userAgentVal)
                .post(requestBody)
                .build();
       Response resp = OkHttpUtils.execute(request);
            if (OK == resp.code()) {
                    JSONObject body = JSON.parseObject(resp.body().string());
                    if ("success".equals(body.getString("status"))) {
                        return true;
                }
            }
        return false;
    }

2, 查询jar包

  Request request = new Request.Builder()
                .url("http://host:port/jars")
                .addHeader(userAgent, userAgentVal)
                .get()
                .build();
  Response response = OkHttpUtils.execute(request);
  String body = response.body().string();

3,提交任务
(特别提示: 提交任务时, Main方法中,容易出现参数解析异常, 为了解决这一个问题, 强烈建议, 对参数进行编码转换, 对programArgs参数进行URLEncoder.encode(参数值, “utf-8”), 然后再在flink运行jar包, 进行解码.

       String baseUrl = "http://host:port/jars/${jarId}/run";
        Map<String, String> params = new HashMap<>();
        params.put("programArgs", "xxxxxx");
        params.put("entryClass", "com.xx.oo.JsonMain");
        params.put("parallelism", "2");
        params.put("savepointPath", null);
        Request request = new Request.Builder()
                .url(baseUrl)
                .addHeader(userAgent, userAgentVal)
                .post(RequestBody.create(JSON.toJSONString(params), MEDIA_TYPE_JSON))
                .build();
        Response resp = OkHttpUtils.execute(request);
        String respBody = resp.body().string();
        if (OK == resp.code()) {
                JSONObject body = JSON.parseObject(respBody);
                return body.getString("jobid");
        }

4,查询任务

       String url= "http://host:port/jobs";
       Request request = new Request.Builder()
                .url(url)
                .addHeader(userAgent, userAgentVal)
                .get()
                .build();
        Response resp = OkHttpUtils.execute(request);
        if (OK == resp.code()) {
            JSONObject body = JSON.parseObject(resp.body().string());
            if (body.containsKey("jobs")) {
                JSONArray jobs = body.getJSONArray("jobs");
                for (int i = 0; i < jobs.size(); i++) {
                    JSONObject jsb = jobs.getJSONObject(i);
                    String id = jsb.getString("id");
                    String status = jsb.getString("status");
                }
            }
        }else{
            logger.error("queryJobByHttp "+resp.body().string());
        }

4,删除任务

       String url= "http://host:port/jobs/${jobId}";
        Request request = new Request.Builder()
                .url(baseUrl)
                .addHeader(userAgent, userAgentVal)
                .patch(RequestBody.create("{}", MEDIA_TYPE_JSON))
                .build();
            Response resp = OkHttpUtils.execute(request);
            if (ACCEPTED == resp.code()) {
                return jobId;
            }
        return null;

flink集群搭建

https://enilu.gitee.io/book/opensource/flink/flink%E9%9B%86%E7%BE%A4%E6%90%AD%E5%BB%BA.html#%E4%B8%8B%E8%BD%BDflink

CarloPan

关注

1
点赞
踩
3

收藏

觉得还不错? 一键收藏
0
评论
Flink REST API #

官方文档：https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/ops/rest_api/https://ci.apache.org/projects/flink/flink-docs-release-1.12/ops/rest_api.html/jarsReturns a list of all jars previously uploaded via '/jars/upload'./jars/uplo.
复制链接

扫一扫