hadoop3.x无法访问50070的原因及解决方法

问题一:
hadoop3.X的webUI已经改到端口 localhost:9870上面,而不是原来的50070

问题二:
Caused by: org.apache.flink.core.fs.UnsupportedFileSystemSchemeException: Could not find a file system implementation for scheme ‘hdfs’. The scheme is not directly supported by Flink and no Hadoop file system to support this scheme could be loaded.

如果要操作hdfs的话,必须要在flink安装目录的 lib 下加上额外的jar包
https://flink.apache.org/downloads.html#apache-flink-1131
下载放到/flink/lib目录中

问题三:
mysql数据库修改时区
时生效,但是当其他人登录时依旧没有修改,重启数据库,发现重启后修改的参数又回到未修改前,在数据库中用命令行修改失败;

可以通过修改配置文件
vim /etc/my.cnf
##在[mysqld]区域中加上
default-time_zone = ‘+8:00’

问题四:
2021-08-04 16:40:45
org.apache.flink.runtime.JobException: Recovery is suppressed by FixedDelayRestartBackoffTimeStrategy(maxNumberRestartAttempts=10, backoffTimeMS=1000)
at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.handleFailure(ExecutionFailureHandler.java:138)
at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.getFailureHandlingResult(ExecutionFailureHandler.java:82)
at org.apache.flink.runtime.scheduler.DefaultScheduler.handleTaskFailure(DefaultScheduler.java:207)
at org.apache.flink.runtime.scheduler.DefaultScheduler.maybeHandleTaskFailure(DefaultScheduler.java:197)
at akka.dispatch.Mailbox.exec(Mailbox.scala:235)
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: org.apache.flink.kafka.shaded.org.apache.kafka.common.KafkaException: Failed to construct kafka producer
at org.apache.flink.kafka.shaded.org.apache.kafka.clients.producer.KafkaProducer.(KafkaProducer.java:432)
at org.apache.flink.kafka.shaded.org.apache.kafka.clients.producer.KafkaProducer.(KafkaProducer.java:298)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:759)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:566)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.flink.kafka.shaded.org.apache.kafka.common.KafkaException: class org.apache.kafka.common.serialization.ByteArraySerializer is not an instance of org.apache.flink.kafka.shaded.org.apache.kafka.common.serialization.Serializer
at org.apache.flink.kafka.shaded.org.apache.kafka.common.config.AbstractConfig.getConfiguredInstance(AbstractConfig.java:374)
at org.apache.flink.kafka.shaded.org.apache.kafka.common.config.AbstractConfig.getConfiguredInstance(AbstractConfig.java:392)
at org.apache.flink.kafka.shaded.org.apache.kafka.clients.producer.KafkaProducer.(KafkaProducer.java:359)
… 24 more

原因:
编写的代码使用的pom是使用
flink-connector-kafka_2.11-1.12.0.jar
对应改jar依赖的是原生的kafka内容,不是shaded内容
但是在flink环境下面,已经提供了
flink-sql-connector-kafka_2.11-1.12.0.jar
可以看到提供的内容,对应进行maven pom文件去掉flink-connector-kafka就可以了,引用flink-sql-connector就可以解决这个问题。

  • 2
    点赞
  • 3
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值