Flink 1.8.0编译:flink-fs-hadoop-shaded找不到

在尝试编译 Flink 1.8.0 时遇到了错误,问题指向 flink-fs-hadoop-shaded 相关模块未找到。目前暂无详细解决步骤,可以参考相关链接 CSDN 博客文章以获取更多线索。

在这里插入图片描述

1.编译报错

[INFO] force-shading ...................................... SUCCESS [  5.703 s]
[INFO] flink 1.8-SNAPSHOT ................................. SUCCESS [ 21.865 s]
[INFO] flink-annotations .................................. SUCCESS [  8.369 s]
[INFO] flink-shaded-hadoop ................................ SUCCESS [  0.890 s]
[INFO] flink-shaded-hadoop2 2.7.6-1.8-SNAPSHOT ............ SUCCESS [ 25.770 s]
[INFO] flink-shaded-hadoop2-uber 2.7.6-1.8-SNAPSHOT ....... SUCCESS [ 32.322 s]
[INFO] flink-shaded-yarn-tests ............................ SUCCESS [ 35.741 s]
[INFO] flink-shaded-curator ............................... SUCCESS [  1.595 s]
[INF
D:\BigData\Idea\JDK\bin\java.exe -Dmaven.multiModuleProjectDirectory=D:\BigData\Idea\Idea创建的文件\FlinkTutorial -Djansi.passthrough=true "-Dmaven.home=D:\BigData\Idea\idea软件信息\IntelliJ IDEA 2024.2.1\plugins\maven\lib\maven3" "-Dclassworlds.conf=D:\BigData\Idea\idea软件信息\IntelliJ IDEA 2024.2.1\plugins\maven\lib\maven3\bin\m2.conf" "-Dmaven.ext.class.path=D:\BigData\Idea\idea软件信息\IntelliJ IDEA 2024.2.1\plugins\maven\lib\maven-event-listener.jar" "-javaagent:D:\BigData\Idea\idea软件信息\IntelliJ IDEA 2024.2.1\lib\idea_rt.jar=54389:D:\BigData\Idea\idea软件信息\IntelliJ IDEA 2024.2.1\bin" -Dfile.encoding=UTF-8 -classpath "D:\BigData\Idea\idea软件信息\IntelliJ IDEA 2024.2.1\plugins\maven\lib\maven3\boot\plexus-classworlds-2.8.0.jar;D:\BigData\Idea\idea软件信息\IntelliJ IDEA 2024.2.1\plugins\maven\lib\maven3\boot\plexus-classworlds.license" org.codehaus.classworlds.Launcher -Didea.version=2024.2.1 package [INFO] Scanning for projects... [INFO] [INFO] ---------------------< org.example:FlinkTutorial >---------------------- [INFO] Building FlinkTutorial 1.0-SNAPSHOT [INFO] from pom.xml [INFO] --------------------------------[ jar ]--------------------------------- [INFO] [INFO] --- resources:3.3.1:resources (default-resources) @ FlinkTutorial --- [INFO] skip non existing resourceDirectory D:\BigData\Idea\Idea创建的文件\FlinkTutorial\src\main\resources [INFO] [INFO] --- compiler:3.13.0:compile (default-compile) @ FlinkTutorial --- [INFO] Nothing to compile - all classes are up to date. [INFO] [INFO] --- resources:3.3.1:testResources (default-testResources) @ FlinkTutorial --- [INFO] skip non existing resourceDirectory D:\BigData\Idea\Idea创建的文件\FlinkTutorial\src\test\resources [INFO] [INFO] --- compiler:3.13.0:testCompile (default-testCompile) @ FlinkTutorial --- [INFO] No sources to compile [INFO] [INFO] --- surefire:3.2.5:test (default-test) @ FlinkTutorial --- [INFO] No tests to run. [INFO] [INFO] --- jar:3.4.1:jar (default-jar) @ FlinkTutorial --- [WARNING] JAR will be empty - no content was marked for inclusion! [INFO] Building jar: D:\BigData\Idea\Idea创建的文件\FlinkTutorial\target\FlinkTutorial-1.0-SNAPSHOT.jar [INFO] [INFO] --- shade:3.2.4:shade (default) @ FlinkTutorial --- [INFO] Including org.apache.flink:flink-streaming-java:jar:1.17.0 in the shaded jar. [INFO] Including org.apache.flink:flink-core:jar:1.17.0 in the shaded jar. [INFO] Including org.apache.flink:flink-annotations:jar:1.17.0 in the shaded jar. [INFO] Including org.apache.flink:flink-metrics-core:jar:1.17.0 in the shaded jar. [INFO] Including org.apache.flink:flink-shaded-asm-9:jar:9.3-16.1 in the shaded jar. [INFO] Including org.apache.flink:flink-shaded-jackson:jar:2.13.4-16.1 in the shaded jar. [INFO] Including org.apache.commons:commons-lang3:jar:3.12.0 in the shaded jar. [INFO] Including org.apache.commons:commons-text:jar:1.10.0 in the shaded jar. [INFO] Including com.esotericsoftware.kryo:kryo:jar:2.24.0 in the shaded jar. [INFO] Including com.esotericsoftware.minlog:minlog:jar:1.2 in the shaded jar. [INFO] Including org.objenesis:objenesis:jar:2.1 in the shaded jar. [INFO] Including commons-collections:commons-collections:jar:3.2.2 in the shaded jar. [INFO] Including org.apache.commons:commons-compress:jar:1.21 in the shaded jar. [INFO] Including org.apache.flink:flink-file-sink-common:jar:1.17.0 in the shaded jar. [INFO] Including org.apache.flink:flink-runtime:jar:1.17.0 in the shaded jar. [INFO] Including org.apache.flink:flink-rpc-core:jar:1.17.0 in the shaded jar. [INFO] Including org.apache.flink:flink-rpc-akka-loader:jar:1.17.0 in the shaded jar. [INFO] Including org.apache.flink:flink-queryable-state-client-java:jar:1.17.0 in the shaded jar. [INFO] Including org.apache.flink:flink-hadoop-fs:jar:1.17.0 in the shaded jar. [INFO] Including commons-io:commons-io:jar:2.11.0 in the shaded jar. [INFO] Including org.apache.flink:flink-shaded-netty:jar:4.1.82.Final-16.1 in the shaded jar. [INFO] Including org.apache.flink:flink-shaded-zookeeper-3:jar:3.7.1-16.1 in the shaded jar. [INFO] Including org.javassist:javassist:jar:3.24.0-GA in the shaded jar. [INFO] Including org.xerial.snappy:snappy-java:jar:1.1.8.3 in the shaded jar. [INFO] Including org.lz4:lz4-java:jar:1.8.0 in the shaded jar. [INFO] Including org.apache.flink:flink-shaded-force-shading:jar:16.1 in the shaded jar. [INFO] Including org.apache.flink:flink-java:jar:1.17.0 in the shaded jar. [INFO] Including com.twitter:chill-java:jar:0.7.6 in the shaded jar. [INFO] Including org.apache.flink:flink-shaded-guava:jar:30.1.1-jre-16.1 in the shaded jar. [INFO] Including org.apache.commons:commons-math3:jar:3.6.1 in the shaded jar. [INFO] Including org.apache.logging.log4j:log4j-slf4j-impl:jar:2.20.0 in the shaded jar. [INFO] Including org.apache.logging.log4j:log4j-api:jar:2.20.0 in the shaded jar. [INFO] Including org.apache.logging.log4j:log4j-core:jar:2.20.0 in the shaded jar. [INFO] Including org.apache.flink:flink-clients:jar:1.17.0 in the shaded jar. [INFO] Including org.apache.flink:flink-optimizer:jar:1.17.0 in the shaded jar. [INFO] Including commons-cli:commons-cli:jar:1.5.0 in the shaded jar. [INFO] Including ch.qos.logback:logback-classic:jar:1.2.11 in the shaded jar. [INFO] Including ch.qos.logback:logback-core:jar:1.2.11 in the shaded jar. [INFO] Excluding org.slf4j:slf4j-api:jar:1.7.36 from the shaded jar. [INFO] Excluding com.google.code.findbugs:jsr305:jar:1.3.9 from the shaded jar. [WARNING] FlinkTutorial-1.0-SNAPSHOT.jar, chill-java-0.7.6.jar, commons-cli-1.5.0.jar, commons-collections-3.2.2.jar, commons-compress-1.21.jar, commons-io-2.11.0.jar, commons-lang3-3.12.0.jar, commons-math3-3.6.1.jar, commons-text-1.10.0.jar, flink-annotations-1.17.0.jar, flink-clients-1.17.0.jar, flink-core-1.17.0.jar, flink-file-sink-common-1.17.0.jar, flink-hadoop-fs-1.17.0.jar, flink-java-1.17.0.jar, flink-metrics-core-1.17.0.jar, flink-optimizer-1.17.0.jar, flink-queryable-state-client-java-1.17.0.jar, flink-rpc-akka-loader-1.17.0.jar, flink-rpc-core-1.17.0.jar, flink-runtime-1.17.0.jar, flink-shaded-asm-9-9.3-16.1.jar, flink-shaded-force-shading-16.1.jar, flink-shaded-guava-30.1.1-jre-16.1.jar, flink-shaded-jackson-2.13.4-16.1.jar, flink-shaded-netty-4.1.82.Final-16.1.jar, flink-shaded-zookeeper-3-3.7.1-16.1.jar, flink-streaming-java-1.17.0.jar, javassist-3.24.0-GA.jar, kryo-2.24.0.jar, log4j-api-2.20.0.jar, log4j-core-2.20.0.jar, log4j-slf4j-impl-2.20.0.jar, logback-classic-1.2.11.jar, logback-core-1.2.11.jar, lz4-java-1.8.0.jar, minlog-1.2.jar, objenesis-2.1.jar, snappy-java-1.1.8.3.jar define 1 overlapping resource: [WARNING] - META-INF/MANIFEST.MF [WARNING] flink-annotations-1.17.0.jar, flink-clients-1.17.0.jar, flink-core-1.17.0.jar, flink-file-sink-common-1.17.0.jar, flink-hadoop-fs-1.17.0.jar, flink-java-1.17.0.jar, flink-metrics-core-1.17.0.jar, flink-optimizer-1.17.0.jar, flink-queryable-state-client-java-1.17.0.jar, flink-rpc-akka-loader-1.17.0.jar, flink-rpc-core-1.17.0.jar, flink-runtime-1.17.0.jar, flink-shaded-asm-9-9.3-16.1.jar, flink-shaded-force-shading-16.1.jar, flink-shaded-guava-30.1.1-jre-16.1.jar, flink-shaded-jackson-2.13.4-16.1.jar, flink-shaded-netty-4.1.82.Final-16.1.jar, flink-shaded-zookeeper-3-3.7.1-16.1.jar, flink-streaming-java-1.17.0.jar, log4j-api-2.20.0.jar, log4j-core-2.20.0.jar, log4j-slf4j-impl-2.20.0.jar, objenesis-2.1.jar define 2 overlapping resources: [WARNING] - META-INF/LICENSE [WARNING] - META-INF/NOTICE [WARNING] flink-annotations-1.17.0.jar, flink-clients-1.17.0.jar, flink-core-1.17.0.jar, flink-file-sink-common-1.17.0.jar, flink-hadoop-fs-1.17.0.jar, flink-java-1.17.0.jar, flink-metrics-core-1.17.0.jar, flink-optimizer-1.17.0.jar, flink-queryable-state-client-java-1.17.0.jar, flink-rpc-akka-loader-1.17.0.jar, flink-rpc-core-1.17.0.jar, flink-runtime-1.17.0.jar, flink-shaded-asm-9-9.3-16.1.jar, flink-shaded-force-shading-16.1.jar, flink-shaded-guava-30.1.1-jre-16.1.jar, flink-shaded-jackson-2.13.4-16.1.jar, flink-shaded-netty-4.1.82.Final-16.1.jar, flink-shaded-zookeeper-3-3.7.1-16.1.jar, flink-streaming-java-1.17.0.jar, log4j-api-2.20.0.jar, log4j-core-2.20.0.jar, log4j-slf4j-impl-2.20.0.jar define 1 overlapping resource: [WARNING] - META-INF/DEPENDENCIES [WARNING] flink-shaded-jackson-2.13.4-16.1.jar, log4j-api-2.20.0.jar define 1 overlapping classes: [WARNING] - META-INF.versions.9.module-info [WARNING] commons-cli-1.5.0.jar, commons-collections-3.2.2.jar, commons-compress-1.21.jar, commons-io-2.11.0.jar, commons-lang3-3.12.0.jar, commons-math3-3.6.1.jar, commons-text-1.10.0.jar define 2 overlapping resources: [WARNING] - META-INF/LICENSE.txt [WARNING] - META-INF/NOTICE.txt [WARNING] flink-shaded-netty-4.1.82.Final-16.1.jar, flink-shaded-zookeeper-3-3.7.1-16.1.jar define 6 overlapping resources: [WARNING] - META-INF/io.netty.versions.properties [WARNING] - META-INF/native-image/io.netty/buffer/native-image.properties [WARNING] - META-INF/native-image/io.netty/common/native-image.properties [WARNING] - META-INF/native-image/io.netty/handler/native-image.properties [WARNING] - META-INF/native-image/io.netty/transport/native-image.properties [WARNING] - META-INF/native-image/io.netty/transport/reflection-config.json [WARNING] log4j-slf4j-impl-2.20.0.jar, logback-classic-1.2.11.jar define 3 overlapping classes: [WARNING] - org.slf4j.impl.StaticLoggerBinder [WARNING] - org.slf4j.impl.StaticMDCBinder [WARNING] - org.slf4j.impl.StaticMarkerBinder [WARNING] maven-shade-plugin has detected that some class files are [WARNING] present in two or more JARs. When this happens, only one [WARNING] single version of the class is copied to the uber jar. [WARNING] Usually this is not harmful and you can skip these warnings, [WARNING] otherwise try to manually exclude artifacts based on [WARNING] mvn dependency:tree -Ddetail=true and the above output. [WARNING] See http://maven.apache.org/plugins/maven-shade-plugin/ [INFO] Replacing original artifact with shaded artifact. [INFO] Replacing D:\BigData\Idea\Idea创建的文件\FlinkTutorial\target\FlinkTutorial-1.0-SNAPSHOT.jar with D:\BigData\Idea\Idea创建的文件\FlinkTutorial\target\FlinkTutorial-1.0-SNAPSHOT-shaded.jar [INFO] Dependency-reduced POM written at: D:\BigData\Idea\Idea创建的文件\FlinkTutorial\dependency-reduced-pom.xml [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 5.862 s [INFO] Finished at: 2025-09-16T23:49:21+08:00 [INFO] ------------------------------------------------------------------------ 进程已结束,退出代码为 0 我这里的报错看一下
最新发布
09-17
oscar@oscardeMacBook-Pro bz-sport-realtime % mvn dependency:tree [INFO] Scanning for projects... [INFO] [INFO] --------------------< com.sport:bz-sport-realtime >--------------------- [INFO] Building bz-sport-realtime 1.0-SNAPSHOT [INFO] from pom.xml [INFO] --------------------------------[ jar ]--------------------------------- [WARNING] 2 problems were encountered while building the effective model for org.apache.yetus:audience-annotations:jar:0.5.0 during dependency collection step for project (use -X to see details) [INFO] [INFO] --- dependency:3.7.0:tree (default-cli) @ bz-sport-realtime --- [INFO] com.sport:bz-sport-realtime:jar:1.0-SNAPSHOT [INFO] +- org.apache.flink:flink-java:jar:1.20.2:compile [INFO] | +- org.apache.commons:commons-lang3:jar:3.12.0:compile [INFO] | +- org.apache.commons:commons-math3:jar:3.6.1:compile [INFO] | +- com.twitter:chill-java:jar:0.7.6:compile [INFO] | +- org.slf4j:slf4j-api:jar:1.7.36:compile [INFO] | \- com.google.code.findbugs:jsr305:jar:1.3.9:compile [INFO] +- org.apache.flink:flink-core:jar:1.20.2:compile [INFO] | +- org.apache.flink:flink-core-api:jar:1.20.2:compile [INFO] | | \- org.apache.flink:flink-metrics-core:jar:1.20.2:compile [INFO] | +- org.apache.flink:flink-annotations:jar:1.20.2:compile [INFO] | +- org.apache.flink:flink-shaded-asm-9:jar:9.5-17.0:compile [INFO] | +- org.apache.flink:flink-shaded-jackson:jar:2.14.2-17.0:compile [INFO] | +- org.snakeyaml:snakeyaml-engine:jar:2.6:compile [INFO] | +- org.apache.commons:commons-text:jar:1.10.0:compile [INFO] | +- com.esotericsoftware.kryo:kryo:jar:2.24.0:compile [INFO] | | +- com.esotericsoftware.minlog:minlog:jar:1.2:compile [INFO] | | \- org.objenesis:objenesis:jar:2.1:compile [INFO] | +- commons-collections:commons-collections:jar:3.2.2:compile [INFO] | +- org.apache.commons:commons-compress:jar:1.26.0:compile [INFO] | \- org.apache.flink:flink-shaded-guava:jar:31.1-jre-17.0:compile [INFO] +- org.apache.flink:flink-connector-base:jar:1.20.2:compile [INFO] +- org.apache.flink:flink-streaming-java:jar:1.20.2:compile [INFO] | +- org.apache.flink:flink-file-sink-common:jar:1.20.2:compile [INFO] | +- org.apache.flink:flink-runtime:jar:1.20.2:compile [INFO] | | +- org.apache.flink:flink-rpc-core:jar:1.20.2:compile [INFO] | | +- org.apache.flink:flink-rpc-akka-loader:jar:1.20.2:compile [INFO] | | +- org.apache.flink:flink-queryable-state-client-java:jar:1.20.2:compile [INFO] | | +- org.apache.flink:flink-hadoop-fs:jar:1.20.2:compile [INFO] | | +- org.apache.flink:flink-shaded-zookeeper-3:jar:3.7.1-17.0:compile [INFO] | | +- org.javassist:javassist:jar:3.24.0-GA:compile [INFO] | | \- tools.profiler:async-profiler:jar:2.9:compile [INFO] | \- org.apache.flink:flink-connector-datagen:jar:1.20.2:compile [INFO] +- org.apache.flink:flink-clients:jar:1.20.2:compile [INFO] | +- org.apache.flink:flink-optimizer:jar:1.20.2:compile [INFO] | +- commons-cli:commons-cli:jar:1.5.0:compile [INFO] | \- org.apache.flink:flink-datastream:jar:1.20.2:compile [INFO] | \- org.apache.flink:flink-datastream-api:jar:1.20.2:compile [INFO] +- org.apache.flink:flink-runtime-web:jar:1.20.2:compile [INFO] | \- org.apache.flink:flink-shaded-netty:jar:4.1.91.Final-17.0:compile [INFO] +- org.apache.flink:flink-connector-kafka:jar:3.4.0-1.20:compile [INFO] | +- com.google.guava:guava:jar:32.1.2-jre:compile [INFO] | | +- com.google.guava:failureaccess:jar:1.0.1:compile [INFO] | | +- com.google.guava:listenablefuture:jar:9999.0-empty-to-avoid-conflict-with-guava:compile [INFO] | | +- org.checkerframework:checker-qual:jar:3.33.0:compile [INFO] | | +- com.google.errorprone:error_prone_annotations:jar:2.18.0:compile [INFO] | | \- com.google.j2objc:j2objc-annotations:jar:2.8:compile [INFO] | +- com.fasterxml.jackson.core:jackson-core:jar:2.15.2:compile [INFO] | +- com.fasterxml.jackson.core:jackson-databind:jar:2.15.2:compile [INFO] | +- com.fasterxml.jackson.datatype:jackson-datatype-jsr310:jar:2.15.2:compile [INFO] | \- com.fasterxml.jackson.datatype:jackson-datatype-jdk8:jar:2.15.2:compile [INFO] +- org.apache.flink:flink-table-api-java:jar:1.20.2:compile [INFO] +- org.apache.flink:flink-table-api-java-bridge:jar:1.20.2:provided [INFO] | \- org.apache.flink:flink-table-api-bridge-base:jar:1.20.2:provided [INFO] +- org.apache.flink:flink-table-planner_2.12:jar:1.20.2:compile [INFO] | +- org.immutables:value:jar:2.8.8:compile [INFO] | +- org.immutables:value-annotations:jar:2.8.8:compile [INFO] | +- org.codehaus.janino:commons-compiler:jar:3.1.10:compile [INFO] | +- org.codehaus.janino:janino:jar:3.1.10:compile [INFO] | +- org.apache.flink:flink-scala_2.12:jar:1.20.2:compile [INFO] | | +- org.scala-lang:scala-reflect:jar:2.12.7:compile [INFO] | | +- org.scala-lang:scala-library:jar:2.12.7:compile [INFO] | | +- org.scala-lang:scala-compiler:jar:2.12.7:compile [INFO] | | | \- org.scala-lang.modules:scala-xml_2.12:jar:1.0.6:compile [INFO] | | \- com.twitter:chill_2.12:jar:0.7.6:compile [INFO] | \- org.apache.flink:flink-table-runtime:jar:1.20.2:compile [INFO] | \- org.apache.flink:flink-cep:jar:1.20.2:compile [INFO] +- org.apache.flink:flink-table-common:jar:1.20.2:compile [INFO] | \- com.ibm.icu:icu4j:jar:67.1:compile [INFO] +- org.apache.doris:flink-doris-connector-1.16:jar:25.1.0:compile [INFO] +- org.apache.flink:flink-connector-jdbc:jar:3.3.0-1.20:compile [INFO] | +- org.apache.flink:flink-connector-jdbc-core:jar:3.3.0-1.20:compile [INFO] | +- org.apache.flink:flink-connector-jdbc-cratedb:jar:3.3.0-1.20:compile [INFO] | +- org.apache.flink:flink-connector-jdbc-db2:jar:3.3.0-1.20:compile [INFO] | +- org.apache.flink:flink-connector-jdbc-mysql:jar:3.3.0-1.20:compile [INFO] | +- org.apache.flink:flink-connector-jdbc-oceanbase:jar:3.3.0-1.20:compile [INFO] | +- org.apache.flink:flink-connector-jdbc-oracle:jar:3.3.0-1.20:compile [INFO] | +- org.apache.flink:flink-connector-jdbc-postgres:jar:3.3.0-1.20:compile [INFO] | +- org.apache.flink:flink-connector-jdbc-sqlserver:jar:3.3.0-1.20:compile [INFO] | \- org.apache.flink:flink-connector-jdbc-trino:jar:3.3.0-1.20:compile [INFO] +- org.apache.flink:flink-connector-mysql-cdc:jar:3.4.0:compile [INFO] | +- org.apache.flink:flink-connector-debezium:jar:3.4.0:compile [INFO] | | +- io.debezium:debezium-api:jar:1.9.8.Final:compile [INFO] | | \- io.debezium:debezium-embedded:jar:1.9.8.Final:compile [INFO] | | +- org.apache.kafka:connect-api:jar:3.2.0:compile [INFO] | | | \- javax.ws.rs:javax.ws.rs-api:jar:2.1.1:runtime [INFO] | | +- org.apache.kafka:connect-runtime:jar:3.2.0:compile [INFO] | | | +- org.apache.kafka:connect-transforms:jar:3.2.0:compile [INFO] | | | +- org.apache.kafka:kafka-tools:jar:3.2.0:runtime [INFO] | | | | \- net.sourceforge.argparse4j:argparse4j:jar:0.7.0:runtime [INFO] | | | +- org.bitbucket.b_c:jose4j:jar:0.7.9:runtime [INFO] | | | +- com.fasterxml.jackson.jaxrs:jackson-jaxrs-json-provider:jar:2.12.6:runtime [INFO] | | | | +- com.fasterxml.jackson.jaxrs:jackson-jaxrs-base:jar:2.12.6:runtime [INFO] | | | | \- com.fasterxml.jackson.module:jackson-module-jaxb-annotations:jar:2.12.6:runtime [INFO] | | | | \- jakarta.xml.bind:jakarta.xml.bind-api:jar:2.3.2:runtime [INFO] | | | +- org.glassfish.jersey.containers:jersey-container-servlet:jar:2.34:runtime [INFO] | | | | +- org.glassfish.jersey.containers:jersey-container-servlet-core:jar:2.34:runtime [INFO] | | | | | \- org.glassfish.hk2.external:jakarta.inject:jar:2.6.1:runtime [INFO] | | | | \- jakarta.ws.rs:jakarta.ws.rs-api:jar:2.1.6:runtime [INFO] | | | +- org.glassfish.jersey.inject:jersey-hk2:jar:2.34:runtime [INFO] | | | | \- org.glassfish.hk2:hk2-locator:jar:2.6.1:runtime [INFO] | | | | +- org.glassfish.hk2.external:aopalliance-repackaged:jar:2.6.1:runtime [INFO] | | | | +- org.glassfish.hk2:hk2-api:jar:2.6.1:runtime [INFO] | | | | \- org.glassfish.hk2:hk2-utils:jar:2.6.1:runtime [INFO] | | | +- javax.activation:activation:jar:1.1.1:compile [INFO] | | | +- org.eclipse.jetty:jetty-servlets:jar:9.4.44.v20210927:runtime [INFO] | | | | \- org.eclipse.jetty:jetty-continuation:jar:9.4.44.v20210927:runtime [INFO] | | | +- org.eclipse.jetty:jetty-client:jar:9.4.44.v20210927:runtime [INFO] | | | +- org.reflections:reflections:jar:0.9.12:runtime [INFO] | | | \- org.apache.maven:maven-artifact:jar:3.8.4:runtime [INFO] | | | \- org.codehaus.plexus:plexus-utils:jar:3.3.0:runtime [INFO] | | +- org.apache.kafka:connect-json:jar:3.2.0:compile [INFO] | | \- org.apache.kafka:connect-file:jar:3.2.0:compile [INFO] | +- io.debezium:debezium-connector-mysql:jar:1.9.8.Final:compile [INFO] | | +- io.debezium:debezium-core:jar:1.9.8.Final:compile [INFO] | | +- io.debezium:debezium-ddl-parser:jar:1.9.8.Final:compile [INFO] | | | \- org.antlr:antlr4-runtime:jar:4.8:compile [INFO] | | +- com.zendesk:mysql-binlog-connector-java:jar:0.27.2:compile [INFO] | | \- mysql:mysql-connector-java:jar:8.0.28:compile [INFO] | +- org.apache.flink:flink-cdc-common:jar:3.4.0:compile [INFO] | +- com.esri.geometry:esri-geometry-api:jar:2.2.0:compile [INFO] | +- com.zaxxer:HikariCP:jar:4.0.3:compile [INFO] | \- org.apache.flink:flink-cdc-runtime:jar:3.4.0:compile [INFO] +- com.mysql:mysql-connector-j:jar:8.0.33:compile [INFO] | \- com.google.protobuf:protobuf-java:jar:3.21.9:compile [INFO] +- org.apache.bahir:flink-connector-redis_2.12:jar:1.1.0:compile [INFO] | +- org.apache.flink:flink-streaming-java_2.12:jar:1.14.5:compile [INFO] | | \- org.apache.flink:flink-shaded-force-shading:jar:14.0:compile [INFO] | \- org.apache.flink:flink-table-api-java-bridge_2.12:jar:1.14.5:compile [INFO] +- redis.clients:jedis:jar:5.2.0:compile [INFO] | +- org.apache.commons:commons-pool2:jar:2.12.0:compile [INFO] | +- org.json:json:jar:20240303:compile [INFO] | \- com.google.code.gson:gson:jar:2.11.0:compile [INFO] +- org.apache.httpcomponents:httpclient:jar:4.5.14:compile [INFO] | +- org.apache.httpcomponents:httpcore:jar:4.4.16:compile [INFO] | +- commons-logging:commons-logging:jar:1.2:compile [INFO] | \- commons-codec:commons-codec:jar:1.11:compile [INFO] +- com.alibaba.fastjson2:fastjson2:jar:2.0.53:compile [INFO] +- org.apache.flink:flink-json:jar:1.20.2:compile [INFO] +- org.projectlombok:lombok:jar:1.18.30:provided [INFO] +- org.apache.hadoop:hadoop-common:jar:3.3.6:compile [INFO] | +- org.apache.hadoop.thirdparty:hadoop-shaded-protobuf_3_7:jar:1.1.1:compile [INFO] | +- org.apache.hadoop:hadoop-annotations:jar:3.3.6:compile [INFO] | +- org.apache.hadoop.thirdparty:hadoop-shaded-guava:jar:1.1.1:compile [INFO] | +- commons-io:commons-io:jar:2.8.0:compile [INFO] | +- commons-net:commons-net:jar:3.9.0:compile [INFO] | +- javax.servlet:javax.servlet-api:jar:3.1.0:compile [INFO] | +- jakarta.activation:jakarta.activation-api:jar:1.2.1:runtime [INFO] | +- org.eclipse.jetty:jetty-server:jar:9.4.51.v20230217:compile [INFO] | | +- org.eclipse.jetty:jetty-http:jar:9.4.51.v20230217:compile [INFO] | | \- org.eclipse.jetty:jetty-io:jar:9.4.51.v20230217:compile [INFO] | +- org.eclipse.jetty:jetty-util:jar:9.4.51.v20230217:compile [INFO] | +- org.eclipse.jetty:jetty-servlet:jar:9.4.51.v20230217:compile [INFO] | | +- org.eclipse.jetty:jetty-security:jar:9.4.51.v20230217:compile [INFO] | | \- org.eclipse.jetty:jetty-util-ajax:jar:9.4.51.v20230217:compile [INFO] | +- org.eclipse.jetty:jetty-webapp:jar:9.4.51.v20230217:compile [INFO] | | \- org.eclipse.jetty:jetty-xml:jar:9.4.51.v20230217:compile [INFO] | +- javax.servlet.jsp:jsp-api:jar:2.1:runtime [INFO] | +- com.sun.jersey:jersey-core:jar:1.19.4:compile [INFO] | | \- javax.ws.rs:jsr311-api:jar:1.1.1:compile [INFO] | +- com.sun.jersey:jersey-servlet:jar:1.19.4:compile [INFO] | +- com.github.pjfanning:jersey-json:jar:1.20:compile [INFO] | | +- org.codehaus.jettison:jettison:jar:1.1:compile [INFO] | | \- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:compile [INFO] | | \- javax.xml.bind:jaxb-api:jar:2.2.2:compile [INFO] | | \- javax.xml.stream:stax-api:jar:1.0-2:compile [INFO] | +- com.sun.jersey:jersey-server:jar:1.19.4:compile [INFO] | +- ch.qos.reload4j:reload4j:jar:1.2.22:compile [INFO] | +- commons-beanutils:commons-beanutils:jar:1.9.4:compile [INFO] | +- org.apache.commons:commons-configuration2:jar:2.8.0:compile [INFO] | +- org.slf4j:slf4j-reload4j:jar:1.7.36:compile [INFO] | +- org.apache.avro:avro:jar:1.7.7:compile [INFO] | | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile [INFO] | | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile [INFO] | | \- com.thoughtworks.paranamer:paranamer:jar:2.3:compile [INFO] | +- com.google.re2j:re2j:jar:1.1:compile [INFO] | +- org.apache.hadoop:hadoop-auth:jar:3.3.6:compile [INFO] | | +- com.nimbusds:nimbus-jose-jwt:jar:9.8.1:compile [INFO] | | | \- com.github.stephenc.jcip:jcip-annotations:jar:1.0-1:compile [INFO] | | +- org.apache.curator:curator-framework:jar:5.2.0:compile [INFO] | | \- org.apache.kerby:kerb-simplekdc:jar:1.0.1:compile [INFO] | | +- org.apache.kerby:kerb-client:jar:1.0.1:compile [INFO] | | | +- org.apache.kerby:kerby-config:jar:1.0.1:compile [INFO] | | | +- org.apache.kerby:kerb-common:jar:1.0.1:compile [INFO] | | | | \- org.apache.kerby:kerb-crypto:jar:1.0.1:compile [INFO] | | | +- org.apache.kerby:kerb-util:jar:1.0.1:compile [INFO] | | | \- org.apache.kerby:token-provider:jar:1.0.1:compile [INFO] | | \- org.apache.kerby:kerb-admin:jar:1.0.1:compile [INFO] | | +- org.apache.kerby:kerb-server:jar:1.0.1:compile [INFO] | | | \- org.apache.kerby:kerb-identity:jar:1.0.1:compile [INFO] | | \- org.apache.kerby:kerby-xdr:jar:1.0.1:compile [INFO] | +- com.jcraft:jsch:jar:0.1.55:compile [INFO] | +- org.apache.curator:curator-client:jar:5.2.0:compile [INFO] | +- org.apache.curator:curator-recipes:jar:5.2.0:compile [INFO] | +- org.apache.zookeeper:zookeeper:jar:3.6.3:compile [INFO] | | +- org.apache.zookeeper:zookeeper-jute:jar:3.6.3:compile [INFO] | | +- org.apache.yetus:audience-annotations:jar:0.5.0:compile [INFO] | | +- io.netty:netty-handler:jar:4.1.63.Final:compile [INFO] | | | +- io.netty:netty-common:jar:4.1.63.Final:compile [INFO] | | | +- io.netty:netty-resolver:jar:4.1.63.Final:compile [INFO] | | | +- io.netty:netty-buffer:jar:4.1.63.Final:compile [INFO] | | | +- io.netty:netty-transport:jar:4.1.63.Final:compile [INFO] | | | \- io.netty:netty-codec:jar:4.1.63.Final:compile [INFO] | | +- io.netty:netty-transport-native-epoll:jar:4.1.63.Final:compile [INFO] | | | \- io.netty:netty-transport-native-unix-common:jar:4.1.63.Final:compile [INFO] | | +- org.slf4j:slf4j-log4j12:jar:1.7.25:compile [INFO] | | \- log4j:log4j:jar:1.2.17:compile [INFO] | +- io.dropwizard.metrics:metrics-core:jar:3.2.4:compile [INFO] | +- org.apache.kerby:kerb-core:jar:1.0.1:compile [INFO] | | \- org.apache.kerby:kerby-pkix:jar:1.0.1:compile [INFO] | | +- org.apache.kerby:kerby-asn1:jar:1.0.1:compile [INFO] | | \- org.apache.kerby:kerby-util:jar:1.0.1:compile [INFO] | +- org.codehaus.woodstox:stax2-api:jar:4.2.1:compile [INFO] | +- com.fasterxml.woodstox:woodstox-core:jar:5.4.0:compile [INFO] | +- dnsjava:dnsjava:jar:2.1.7:compile [INFO] | \- org.xerial.snappy:snappy-java:jar:1.1.8.2:compile [INFO] +- org.apache.hadoop:hadoop-hdfs-client:jar:3.3.6:compile [INFO] | +- com.squareup.okhttp3:okhttp:jar:4.9.3:compile [INFO] | | \- com.squareup.okio:okio:jar:2.8.0:compile [INFO] | +- org.jetbrains.kotlin:kotlin-stdlib:jar:1.4.10:compile [INFO] | +- org.jetbrains.kotlin:kotlin-stdlib-common:jar:1.4.10:compile [INFO] | \- com.fasterxml.jackson.core:jackson-annotations:jar:2.12.7:compile [INFO] \- org.apache.kafka:kafka-clients:jar:3.6.0:compile [INFO] +- com.github.luben:zstd-jni:jar:1.5.5-1:compile [INFO] \- org.lz4:lz4-java:jar:1.8.0:compile [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 0.779 s [INFO] Finished at: 2025-08-27T18:32:41+08:00 [INFO] ------------------------------------------------------------------------ oscar@oscardeMacBook-Pro bz-sport-realtime % 我的树是这样,那么你说的依赖冲突不存在,为什么还是会报我本地idea上flink任务可以正常消费Kafka,但是上yarn集群就报如下错误: org.apache.kafka.common.KafkaException: Failed to construct kafka consumer at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:830) ~[bz-sport-realtime-1.0-SNAPSHOT.jar:?] at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:665) ~[bz-sport-realtime-1.0-SNAPSHOT.jar:?] at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:646) ~[bz-sport-realtime-1.0-SNAPSHOT.jar:?] at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:626) ~[bz-sport-realtime-1.0-SNAPSHOT.jar:?] at org.apache.flink.connector.kafka.source.reader.KafkaPartitionSplitReader.<init>(KafkaPartitionSplitReader.java:97) ~[bz-sport-realtime-1.0-SNAPSHOT.jar:?] at org.apache.flink.connector.kafka.source.KafkaSource.lambda$createReader$1(KafkaSource.java:185) ~[bz-sport-realtime-1.0-SNAPSHOT.jar:?] at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager.createSplitFetcher(SplitFetcherManager.java:259) ~[bz-sport-realtime-1.0-SNAPSHOT.jar:?] at org.apache.flink.connector.base.source.reader.fetcher.SingleThreadFetcherManager.addSplits(SingleThreadFetcherManager.java:148) ~[bz-sport-realtime-1.0-SNAPSHOT.jar:?] at org.apache.flink.connector.base.source.reader.SourceReaderBase.addSplits(SourceReaderBase.java:315) ~[bz-sport-realtime-1.0-SNAPSHOT.jar:?] at org.apache.flink.streaming.api.operators.SourceOperator.handleAddSplitsEvent(SourceOperator.java:626) ~[bz-sport-realtime-1.0-SNAPSHOT.jar:?] at org.apache.flink.streaming.api.operators.SourceOperator.handleOperatorEvent(SourceOperator.java:596) ~[bz-sport-realtime-1.0-SNAPSHOT.jar:?] at org.apache.flink.streaming.runtime.tasks.OperatorEventDispatcherImpl.dispatchEventToHandlers(OperatorEventDispatcherImpl.java:72) ~[bz-sport-realtime-1.0-SNAPSHOT.jar:?] at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.dispatchOperatorEvent(RegularOperatorChain.java:80) ~[bz-sport-realtime-1.0-SNAPSHOT.jar:?] at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$dispatchOperatorEvent$24(StreamTask.java:1609) ~[bz-sport-realtime-1.0-SNAPSHOT.jar:?] at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50) ~[bz-sport-realtime-1.0-SNAPSHOT.jar:?] at org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:101) ~[bz-sport-realtime-1.0-SNAPSHOT.jar:?] at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMail(MailboxProcessor.java:414) ~[bz-sport-realtime-1.0-SNAPSHOT.jar:?] at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsWhenDefaultActionUnavailable(MailboxProcessor.java:383) ~[bz-sport-realtime-1.0-SNAPSHOT.jar:?] at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:368) ~[bz-sport-realtime-1.0-SNAPSHOT.jar:?] at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:229) ~[bz-sport-realtime-1.0-SNAPSHOT.jar:?] at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:973) ~[bz-sport-realtime-1.0-SNAPSHOT.jar:?] at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:917) ~[bz-sport-realtime-1.0-SNAPSHOT.jar:?] at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:970) ~[bz-sport-realtime-1.0-SNAPSHOT.jar:?] at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:949) [bz-sport-realtime-1.0-SNAPSHOT.jar:?] at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:763) [bz-sport-realtime-1.0-SNAPSHOT.jar:?] at org.apache.flink.runtime.taskmanager.Task.run(Task.java:575) [bz-sport-realtime-1.0-SNAPSHOT.jar:?] at java.lang.Thread.run(Thread.java:750) [?:1.8.0_451] Caused by: org.apache.kafka.common.KafkaException: class org.apache.kafka.common.serialization.ByteArrayDeserializer is not an instance of org.apache.kafka.common.serialization.Deserializer at org.apache.kafka.common.config.AbstractConfig.getConfiguredInstance(AbstractConfig.java:405) ~[bz-sport-realtime-1.0-SNAPSHOT.jar:?] at org.apache.kafka.common.config.AbstractConfig.getConfiguredInstance(AbstractConfig.java:436) ~[bz-sport-realtime-1.0-SNAPSHOT.jar:?] at org.apache.kafka.common.config.AbstractConfig.getConfiguredInstance(AbstractConfig.java:421) ~[bz-sport-realtime-1.0-SNAPSHOT.jar:?] at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:709) ~[bz-sport-realtime-1.0-SNAPSHOT.jar:?] ... 26 more
08-28
评论 1
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
红包 添加红包
表情包 插入表情
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

九师兄

你的鼓励是我做大写作的动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值