pyflink部署报错：Cannot instantiate the coordinator for operator Source: xxx

于 2024-04-10 09:44:46 发布
阅读量325 收藏 5
点赞数 4
文章标签： flink
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/m0_56161419/article/details/137582565
版权

今天测试pyflink的任务，pycharm本地测试完以后进入linux进行standalone模式部署。

缺少kafka的包，上传上去遇到了这个报错。

先说解决方法：重启集群

通常这个报错下面还会有一个java.lang.ClassNotFoundException: org.apache.flink.connector.kafka.source.KafkaSource

因为和无kafka包的报错不太一样所以没去注意ClassNotFound，把重心放在了标题的报错：Cannot instantiate the coordinator for operator Source: kafka_source_from_topic_FlightDynData_From_Gaodu[1] -> Correlate[2] -> Calc[3] -> ods_FlightDynData_kafka_target[4]: Writer -> ods_FlightDynData_kafka_target[4]: Committer

以下为示例：

无kafka包时部署报错

Traceback (most recent call last):
  File "/root/anaconda3/envs/pyflink_38/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/root/anaconda3/envs/pyflink_38/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/tmp/pyflink/e47ee5f6-d68d-40cf-88a7-ab649e9da9c0/3ad4485f-8aff-4f46-8959-154e03bebd5b/FlightDynDataFromGaodu.py", line 230, in <module>
    jobFlightDynDataFromGaodu()
  File "/tmp/pyflink/e47ee5f6-d68d-40cf-88a7-ab649e9da9c0/3ad4485f-8aff-4f46-8959-154e03bebd5b/FlightDynDataFromGaodu.py", line 58, in jobFlightDynDataFromGaodu
    table_env.execute_sql("""
  File "/opt/flink-1.16.1/opt/python/pyflink.zip/pyflink/table/table_environment.py", line 836, in execute_sql
  File "/opt/flink-1.16.1/opt/python/py4j-0.10.9.3-src.zip/py4j/java_gateway.py", line 1321, in __call__
  File "/opt/flink-1.16.1/opt/python/pyflink.zip/pyflink/util/exceptions.py", line 146, in deco
  File "/opt/flink-1.16.1/opt/python/py4j-0.10.9.3-src.zip/py4j/protocol.py", line 326, in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling o8.executeSql.
: org.apache.flink.table.api.ValidationException: Unable to create a source for reading table 'default_catalog.default_database.kafka_source_from_topic_FlightDynData_From_Gaodu'.

Table options are:

'connector'='kafka'
'format'='json'
'json.fail-on-missing-field'='false'
'json.ignore-parse-errors'='true'
'properties.bootstrap.servers'='192.168.87.101:9092,192.168.87.102:9092,192.168.87.103:9092'
'properties.group.id'='test_group_kafka_topic'
'scan.startup.mode'='latest-offset'
'topic'='test-topic'
        at org.apache.flink.table.factories.FactoryUtil.createDynamicTableSource(FactoryUtil.java:166)
        at org.apache.flink.table.factories.FactoryUtil.createDynamicTableSource(FactoryUtil.java:191)
        at org.apache.flink.table.planner.plan.schema.CatalogSourceTable.createDynamicTableSource(CatalogSourceTable.java:175)
        at org.apache.flink.table.planner.plan.schema.CatalogSourceTable.toRel(CatalogSourceTable.java:115)
        at org.apache.calcite.sql2rel.SqlToRelConverter.toRel(SqlToRelConverter.java:3619)
        at org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2559)
        at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2175)
        at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2095)
        at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2038)
        at org.apache.calcite.sql2rel.SqlToRelConverter.convertJoin(SqlToRelConverter.java:2902)
        at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2194)
        at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2038)
        at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:669)
        at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:657)
        at org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3462)
        at org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:570)
        at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.org$apache$flink$table$planner$calcite$FlinkPlannerImpl$$rel(FlinkPlannerImpl.scala:215)
        at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.rel(FlinkPlannerImpl.scala:191)
        at org.apache.flink.table.planner.operations.SqlToOperationConverter.toQueryOperation(SqlToOperationConverter.java:1498)
        at org.apache.flink.table.planner.operations.SqlToOperationConverter.convertViewQuery(SqlToOperationConverter.java:1128)
        at org.apache.flink.table.planner.operations.SqlToOperationConverter.convertCreateView(SqlToOperationConverter.java:1095)
        at org.apache.flink.table.planner.operations.SqlToOperationConverter.convertValidatedSqlNode(SqlToOperationConverter.java:319)
        at org.apache.flink.table.planner.operations.SqlToOperationConverter.convert(SqlToOperationConverter.java:262)
        at org.apache.flink.table.planner.delegation.ParserImpl.parse(ParserImpl.java:106)
        at org.apache.flink.table.api.internal.TableEnvironmentImpl.executeSql(TableEnvironmentImpl.java:723)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at org.apache.flink.api.python.shaded.py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
        at org.apache.flink.api.python.shaded.py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
        at org.apache.flink.api.python.shaded.py4j.Gateway.invoke(Gateway.java:282)
        at org.apache.flink.api.python.shaded.py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
        at org.apache.flink.api.python.shaded.py4j.commands.CallCommand.execute(CallCommand.java:79)
        at org.apache.flink.api.python.shaded.py4j.GatewayConnection.run(GatewayConnection.java:238)
        at java.base/java.lang.Thread.run(Thread.java:832)
Caused by: org.apache.flink.table.api.ValidationException: Cannot discover a connector using option: 'connector'='kafka'
        at org.apache.flink.table.factories.FactoryUtil.enrichNoMatchingConnectorError(FactoryUtil.java:735)
        at org.apache.flink.table.factories.FactoryUtil.discoverTableFactory(FactoryUtil.java:709)
        at org.apache.flink.table.factories.FactoryUtil.createDynamicTableSource(FactoryUtil.java:162)
        ... 35 more
Caused by: org.apache.flink.table.api.ValidationException: Could not find any factory for identifier 'kafka' that implements 'org.apache.flink.table.factories.DynamicTableFactory' in the classpath.

Available factory identifiers are:

blackhole
datagen
filesystem
print
python-input-format
        at org.apache.flink.table.factories.FactoryUtil.discoverFactory(FactoryUtil.java:545)
        at org.apache.flink.table.factories.FactoryUtil.enrichNoMatchingConnectorError(FactoryUtil.java:731)
        ... 37 more

org.apache.flink.client.program.ProgramAbortException: java.lang.RuntimeException: Python process exits with code: 1
        at org.apache.flink.client.python.PythonDriver.main(PythonDriver.java:140)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355)
        at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222)
        at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:98)
        at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:843)
        at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:240)
        at org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:1087)
        at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1165)
        at org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:28)
        at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1165)
Caused by: java.lang.RuntimeException: Python process exits with code: 1
        at org.apache.flink.client.python.PythonDriver.main(PythonDriver.java:130)
        ... 13 more

上传包之后报错

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.apache.flink.api.java.ClosureCleaner (file:/opt/flink-1.16.1/lib/flink-dist-1.16.1.jar) to field java.lang.String.value
WARNING: Please consider reporting this to the maintainers of org.apache.flink.api.java.ClosureCleaner
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Traceback (most recent call last):
  File "/root/anaconda3/envs/pyflink_38/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/root/anaconda3/envs/pyflink_38/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/tmp/pyflink/0b6a6c0b-b284-4425-b83f-4dea78e38f10/3a81bfd3-0105-444d-930e-78f4e009f4e3/FlightDynDataFromGaodu.py", line 230, in <module>
    jobFlightDynDataFromGaodu()
  File "/tmp/pyflink/0b6a6c0b-b284-4425-b83f-4dea78e38f10/3a81bfd3-0105-444d-930e-78f4e009f4e3/FlightDynDataFromGaodu.py", line 180, in jobFlightDynDataFromGaodu
    table_env.execute_sql("""
  File "/opt/flink-1.16.1/opt/python/pyflink.zip/pyflink/table/table_environment.py", line 836, in execute_sql
  File "/opt/flink-1.16.1/opt/python/py4j-0.10.9.3-src.zip/py4j/java_gateway.py", line 1321, in __call__
  File "/opt/flink-1.16.1/opt/python/pyflink.zip/pyflink/util/exceptions.py", line 158, in deco
pyflink.util.exceptions.TableException: org.apache.flink.table.api.TableException: Failed to execute sql
        at org.apache.flink.table.api.internal.TableEnvironmentImpl.executeInternal(TableEnvironmentImpl.java:867)
        at org.apache.flink.table.api.internal.TableEnvironmentImpl.executeInternal(TableEnvironmentImpl.java:827)
        at org.apache.flink.table.api.internal.TableEnvironmentImpl.executeInternal(TableEnvironmentImpl.java:918)
        at org.apache.flink.table.api.internal.TableEnvironmentImpl.executeSql(TableEnvironmentImpl.java:730)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at org.apache.flink.api.python.shaded.py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
        at org.apache.flink.api.python.shaded.py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
        at org.apache.flink.api.python.shaded.py4j.Gateway.invoke(Gateway.java:282)
        at org.apache.flink.api.python.shaded.py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
        at org.apache.flink.api.python.shaded.py4j.commands.CallCommand.execute(CallCommand.java:79)
        at org.apache.flink.api.python.shaded.py4j.GatewayConnection.run(GatewayConnection.java:238)
        at java.base/java.lang.Thread.run(Thread.java:832)
Caused by: org.apache.flink.util.FlinkException: Failed to execute job 'pyflink-job'.
        at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.executeAsync(StreamExecutionEnvironment.java:2203)
        at org.apache.flink.client.program.StreamContextEnvironment.executeAsync(StreamContextEnvironment.java:206)
        at org.apache.flink.table.planner.delegation.DefaultExecutor.executeAsync(DefaultExecutor.java:95)
        at org.apache.flink.table.executor.python.ChainingOptimizingExecutor.executeAsync(ChainingOptimizingExecutor.java:73)
        at org.apache.flink.table.api.internal.TableEnvironmentImpl.executeInternal(TableEnvironmentImpl.java:850)
        ... 14 more
Caused by: java.lang.RuntimeException: org.apache.flink.runtime.client.JobInitializationException: Could not start the JobMaster.
        at org.apache.flink.util.ExceptionUtils.rethrow(ExceptionUtils.java:321)
        at org.apache.flink.util.function.FunctionUtils.lambda$uncheckedFunction$2(FunctionUtils.java:75)
        at java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:642)
        at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:479)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
        at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1016)
        at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1665)
        at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1598)
        at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
Caused by: org.apache.flink.runtime.client.JobInitializationException: Could not start the JobMaster.
        at org.apache.flink.runtime.jobmaster.DefaultJobMasterServiceProcess.lambda$new$0(DefaultJobMasterServiceProcess.java:97)
        at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)
        at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837)
        at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
        at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1769)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
        at java.base/java.lang.Thread.run(Thread.java:832)
Caused by: java.util.concurrent.CompletionException: java.lang.RuntimeException: org.apache.flink.runtime.JobException: Cannot instantiate the coordinator for operator Source: kafka_source_from_topic_FlightDynData_From_Gaodu[1] -> Correlate[2] -> Calc[3] -> ods_FlightDynData_kafka_target[4]: Writer -> ods_FlightDynData_kafka_target[4]: Committer
        at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:314)
        at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:319)
        at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1766)
        ... 3 more
Caused by: java.lang.RuntimeException: org.apache.flink.runtime.JobException: Cannot instantiate the coordinator for operator Source: kafka_source_from_topic_FlightDynData_From_Gaodu[1] -> Correlate[2] -> Calc[3] -> ods_FlightDynData_kafka_target[4]: Writer -> ods_FlightDynData_kafka_target[4]: Committer
        at org.apache.flink.util.ExceptionUtils.rethrow(ExceptionUtils.java:321)
        at org.apache.flink.util.function.FunctionUtils.lambda$uncheckedSupplier$4(FunctionUtils.java:114)
        at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1764)
        ... 3 more
Caused by: org.apache.flink.runtime.JobException: Cannot instantiate the coordinator for operator Source: kafka_source_from_topic_FlightDynData_From_Gaodu[1] -> Correlate[2] -> Calc[3] -> ods_FlightDynData_kafka_target[4]: Writer -> ods_FlightDynData_kafka_target[4]: Committer
        at org.apache.flink.runtime.executiongraph.ExecutionJobVertex.initialize(ExecutionJobVertex.java:229)
        at org.apache.flink.runtime.executiongraph.DefaultExecutionGraph.initializeJobVertex(DefaultExecutionGraph.java:901)
        at org.apache.flink.runtime.executiongraph.DefaultExecutionGraph.initializeJobVertices(DefaultExecutionGraph.java:891)
        at org.apache.flink.runtime.executiongraph.DefaultExecutionGraph.attachJobGraph(DefaultExecutionGraph.java:848)
        at org.apache.flink.runtime.executiongraph.DefaultExecutionGraph.attachJobGraph(DefaultExecutionGraph.java:830)
        at org.apache.flink.runtime.executiongraph.DefaultExecutionGraphBuilder.buildGraph(DefaultExecutionGraphBuilder.java:203)
        at org.apache.flink.runtime.scheduler.DefaultExecutionGraphFactory.createAndRestoreExecutionGraph(DefaultExecutionGraphFactory.java:156)
        at org.apache.flink.runtime.scheduler.SchedulerBase.createAndRestoreExecutionGraph(SchedulerBase.java:361)
        at org.apache.flink.runtime.scheduler.SchedulerBase.<init>(SchedulerBase.java:206)
        at org.apache.flink.runtime.scheduler.DefaultScheduler.<init>(DefaultScheduler.java:134)
        at org.apache.flink.runtime.scheduler.DefaultSchedulerFactory.createInstance(DefaultSchedulerFactory.java:152)
        at org.apache.flink.runtime.jobmaster.DefaultSlotPoolServiceSchedulerFactory.createScheduler(DefaultSlotPoolServiceSchedulerFactory.java:119)
        at org.apache.flink.runtime.jobmaster.JobMaster.createScheduler(JobMaster.java:369)
        at org.apache.flink.runtime.jobmaster.JobMaster.<init>(JobMaster.java:346)
        at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.internalCreateJobMasterService(DefaultJobMasterServiceFactory.java:123)
        at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.lambda$createJobMasterService$0(DefaultJobMasterServiceFactory.java:95)
        at org.apache.flink.util.function.FunctionUtils.lambda$uncheckedSupplier$4(FunctionUtils.java:112)
        ... 4 more
Caused by: java.lang.ClassNotFoundException: org.apache.flink.connector.kafka.source.KafkaSource
        at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:435)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
        at org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:67)
        at org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:51)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
        at org.apache.flink.util.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.loadClass(FlinkUserCodeClassLoaders.java:192)
        at java.base/java.lang.Class.forName0(Native Method)
        at java.base/java.lang.Class.forName(Class.java:468)
        at org.apache.flink.util.InstantiationUtil$ClassLoaderObjectInputStream.resolveClass(InstantiationUtil.java:78)
        at java.base/java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:2028)
        at java.base/java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1895)
        at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2202)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1712)
        at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2540)
        at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2434)
        at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2235)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1712)
        at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:519)
        at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:477)
        at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:617)
        at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:602)
        at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:589)
        at org.apache.flink.util.SerializedValue.deserializeValue(SerializedValue.java:67)
        at org.apache.flink.runtime.operators.coordination.OperatorCoordinatorHolder.create(OperatorCoordinatorHolder.java:488)
        at org.apache.flink.runtime.executiongraph.ExecutionJobVertex.createOperatorCoordinatorHolder(ExecutionJobVertex.java:286)
        at org.apache.flink.runtime.executiongraph.ExecutionJobVertex.initialize(ExecutionJobVertex.java:223)
        ... 20 more

org.apache.flink.client.program.ProgramAbortException: java.lang.RuntimeException: Python process exits with code: 1
        at org.apache.flink.client.python.PythonDriver.main(PythonDriver.java:140)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355)
        at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222)
        at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:98)
        at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:843)
        at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:240)
        at org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:1087)
        at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1165)
        at org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:28)
        at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1165)
Caused by: java.lang.RuntimeException: Python process exits with code: 1
        at org.apache.flink.client.python.PythonDriver.main(PythonDriver.java:130)
        ... 13 more

重启集群后部署成功

[root@k8s-master bin]# ./flink run --jobmanager 192.168.87.101:8081 -pyexec /root/anaconda3/envs/pyflink_38/bin/python3 -pyclientexec /root/anaconda3/envs/pyflink_38/bin/python3 -py /opt/testDir/dikong/FlightDynDataFromGaodu.py --conf /opt/testDir/etc/config.toml
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.apache.flink.api.java.ClosureCleaner (file:/opt/flink-1.16.1/lib/flink-dist-1.16.1.jar) to field java.lang.String.value
WARNING: Please consider reporting this to the maintainers of org.apache.flink.api.java.ClosureCleaner
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Job has been submitted with JobID 25da1e32406b24239b9139699b8e56fa
Job has been submitted with JobID 25f9fcca4b55139b06b3187390f936b3

muce
关注
  • 4
    点赞
  • 踩
  • 5
    收藏
    觉得还不错? 一键收藏
  • 1
    评论
【Flink 实战系列】Could not find any factory for identifier ‘wechat-alarm‘ that implements ‘org.apache.fli
JasonLee实时计算
05-10 517
但是我确实把相关的代码都打到最终的 jar 包里了，还是遇到了这个问题，这是咋回事呢？大家都知道 Flink 是通过 SPI 机制来识别这些 connector 的，那么就需要添加 META-INF/services 相关的配置文件，我也添加了 META-INF/services/org.apache.flink.table.factories.Factory 这个文件，并且我在本地已经测试过 wechat-alarm connector 是可以正常使用的，但是打完包放到集群上就不行了呢？
Flink-CDC Cannot instantiate the coordinator for operator Source
boatingMan的博客
02-21 1687
flink-cdc加载表遇到问题
Flink Caused by: org.apache.flink.runtime.JobException:错误
zhangyupeng0528的博客
11-08 1万+
这里写自定义目录标题欢迎使用Markdown编辑器新的改变功能快捷键合理的创建标题，有助于目录的生成如何改变文本的样式插入链接与图片如何插入一段漂亮的代码片生成一个适合你的列表创建一个表格设定内容居中、居左、居右SmartyPants创建一个自定义列表如何创建一个注脚注释也是必不可少的KaTeX数学公式新的甘特图功能，丰富你的文章UML 图表FLowchart流程图导出与导入导出导入 欢迎使用Markdown编辑器 你好！ 这是你第一次使用 Markdown编辑器 所展示的欢迎页。如果你想学习如何使用Mar
【已解决】Flink连接JDBC报错 org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
qq_31412425的博客
06-10 3407
true修改为false即可。集成环境Idea2020。
flink常见报错及解决方案
吴成伟的博客
05-11 9003
Exception in thread "main" org.apache.flink.runtime.client.JobExecutionException: Job execution failed. Caused by: java.lang.Exception: java.net.SocketException: Connection reset Caused by: java.net.SocketException: Connection reset No new data sinks have
Unity3D教程：Prefabs与Instantiate1
08-08
Unity3D教程：Prefabs与Instantiate1
php-ovh-sms:PHP for OVH短信API
05-22
* # Instantiate. Visit https://api.ovh.com/createToken/index.cgi?GET=/sms&GET=/sms/*&PUT=/sms/*&DELETE=/sms/*&POST=/sms/* * to get your credentials */ require __DIR__ . '/vendor/autoload.php' ; use...
Unity3D入门 :Prefabs(预设)与 Instantiate(实例化)
04-01
学习使用Prefabs和instantiate使用方法！
react-wasm:用于React的声明性WebAssembly实例
05-02
React堆 用于React的声明性WebAssembly实例安装您可以使用安装react-wasm： npm install --save react-wasm 如果您不在项目中使用npm，则可以使用dist文件夹中带有[removed]标记的UMD build来包含reactWasm。...
hewy:你多早？
03-11
HEWY（您多早） 主网地址： terra1d0xll49762dc8dzz4tgrum7jxxhgsx2lawfmf7网页界面： ： 背景 资料来源： : 部署日志在里面height: 2127830txhash: DCD749B7D4DB976E0477502FB0C958D3529B9BB19BE2DEAA9E3EB3EE8871...
在idea中测试flink的流批处理时,读文件时出现错误
zaijainruqiu的博客
08-12 984
idea无法读取相对路径文件
flink本地模式运行错误Caused by: org.apache.flink.runtime.JobException: Recovery is suppressed by NoRestartB
weixin_52116589的博客
05-07 7773
Flink 本地模式运行错误 Exception in thread "main" org.apache.flink.runtime.client.JobExecutionException: Job execution failed. Caused by: org.apache.flink.runtime.JobException: Recovery is suppressed by NoRestartBackoffTimeStrategy Caused by: java.net.ConnectExce
IDEA中运行Dinky0.7.5之KAFKA数据源调试问题解决过程
最新发布
u010782920的博客
12-01 1578
IDEA中运行Dinky0.7.5之KAFKA数据源调试问题解决过程 java.lang.ClassCastException: cannot assign instance of org.apache.kafka.clients.consumer.OffsetResetStrategy to org.apache.flink.kafka.shaded.org.apache.kafka.clients.consumer.OffsetResetStrategy
4.3、Flink任务怎样读取Kafka中的数据
向热爱自己一样，热爱这个世界。
08-11 5298
flink1.17 读取kafka 数据
【Flink异常】Could not find any factory for identifier ‘jdbc‘
热门推荐
search-lemon的博客
06-22 1万+
文章目录1. 异常详情2. 原因3. 解决办法 笔者在项目中添加了flink-connector-jdbc的依赖，全局搜索org.apache.flink.table.factories.DynamicTableFactory的实现类中有很多，其中包含Jdbc的实现类，但在服务器上运行任务时却报以下错误。 1. 异常详情 Caused by: org.apache.flink.table.api.ValidationException: Could not find any factory for iden
flink1.18.0 sql-client报错
yy的博客
11-09 1136
【代码】flink1.18.0 sql-client报错。
java Flink（十七）flink程序打包 单机部署 以及与kafka遇到的那些问题 解决 NoClassDefFoundError 解决无法消费 解决无法加载类用户classloader
qq_40771567的博客
05-08 3467
今天记录一下这两天部署flink程序的过程以及遇到的坑。 首先我的服务是一个flink计算服务，里边用到了kafka、redis等。 一、flink下载安装 https://archive.apache.org/dist/flink/ 点击下载flink版本，这里要选择好自己的flink匹配的hadoop版本以及scala版本。 这里我们选择的是比较稳定的1.7.2 ...
Flink任务失败，检查点失效：Exceeded checkpoint tolerable failure threshold.
weixin_45943866的博客
07-19 1万+
在报Exceeded checkpoint tolerable failure threshold.错误的之前，是先报的是Checkpoint expired before completing.大概意思是检查点在完成前过期了。
qt报出qcoreapplication applicationdirpath : please instantiate the qapplication object first怎么解决
04-29
这个错误通常是因为你在使用 `QCoreApplication::applicationDirPath()` 函数时没有先实例化 `QApplication` 对象。解决方法是在使用 `QCoreApplication::applicationDirPath()` 函数之前，先实例化一个 `...

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 1
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值