Flink WordCount 报错

最新推荐文章于 2023-06-10 16:26:02 发布

weixin_43843739

最新推荐文章于 2023-06-10 16:26:02 发布

阅读量1.6k

点赞数

分类专栏： Flink

本文链接：https://blog.csdn.net/weixin_43843739/article/details/102683985

版权

本文档描述了在使用Flink WordCount Java API时遇到的错误，问题在于无法读取到数据文件'words'。错误信息提示文件不存在或者用户权限不足。解决方法是将'data'目录放置在代码的上一级目录。

摘要由CSDN通过智能技术生成

在这里插入图片描述

Flink WordCount JavaAPI（从文件读取）
图中，data目录下是要读取的文件：words
内容：
hello java
hello spark
hello spark
hello spark
hello flink
hello flink
hello flink
hello flink
hello hadoop
hello hadoop

popx.xml文件：

代码：
import org.apache.flink.api.common.functions.FlatMapFunction;
import org.apache.flink.api.common.functions.MapFunction;
import org.apache.flink.api.java.ExecutionEnvironment;
import org.apache.flink.api.java.operators.*;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.util.Collector;

public class Test1 {

public static void main(String[] args) throws Exception {
    //final StreamExecutionEnvironment  environment = StreamExecutionEnvironment .getExecutionEnvironment();
    //DataStream<String> text = environment.readTextFile("./data/worlds");
    ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    DataSource<String> dataSource = env.readTextFile("src/main/data/worlds");
    FlatMapOperator<String, String> words = dataSource.flatMap(new FlatMapFunction<String, String>() {
        @Override
        public void flatMap(String line, Collector<String> collector) throws Exception {
            String[] split = line.split(" ");
            for (String word : split) {
                collector.collect(word);
            }
        }
    });

    MapOperator<String, Tuple2<String, Integer>> pairWords = words.map(new MapFunction<String, Tuple2<String, Integer>>() {
        @Override
        public Tuple2<String, Integer> map(String word) throws Exception {
            return new Tuple2<>(word, 1);
        }
    });

    UnsortedGrouping<Tuple2<String, Integer>> tuple2UnsortedGrouping = pairWords.groupBy(0);
    AggregateOperator<Tuple2<String, Integer>> sum = tuple2UnsortedGrouping.sum(1);
    sum.print();

}

}

第一次运行报错：
Exception in thread “main” org.apache.flink.runtime.client.JobExecutionException: Could not retrieve JobResult.
at org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:643)
at org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:223)
at org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:91)
at org.apache.flink.api.java.ExecutionEnvironment.execute(ExecutionEnvironment.java:817)
at org.apache.flink.api.java.DataSet.collect(DataSet.java:413)
at org.apache.flink.api.java.DataSet.print(DataSet.java:1652)
at Test1.main(Test1.java:34)
Caused by: org.apache.flink.runtime.client.JobSubmissionException: Failed to submit job.
at org.apache.flink.runtime.dispatcher.Dispatcher.lambda$submitJob $2 (D i s p a t c h e r . j a v a : 267) a t j a v a . u t i l . c o n c u r r e n t . C o m p l e t a b l e F u t u r e . u n i E x c e p t i o n a l l y (C o m p l e t a b l e F u t u r e . j a v a : 870) a t j a v a . u t i l . c o n c u r r e n t . C o m p l e t a b l e F u t u r e$ UniExceptionally.tryFire(CompletableFuture.java:852)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.postFire(CompletableFuture.java:561)
at java.util.concurrent.CompletableFuture $U n i W h e n C o m p l e t e . t r y F$

最低0.47元/天解锁文章

weixin_43843739

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
Flink WordCount 报错

Flink WordCount JavaAPI（从文件读取）图中，data目录下是要读取的文件：words内容：hello javahello sparkhello sparkhello sparkhello flinkhello flinkhello flinkhello flinkhello hadoophello hadooppopx.xml文件：代码：im...
复制链接

扫一扫

专栏目录