xinji

代码块 StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment(); StreamTableEnvironment tEnv = StreamTableEnvironment.create(env); DataStreamSource<PageVisit> input = env.fromElements( new P

在使用新版本的hadoop3.2.2时,启动spark sql 发现以下报错Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.security.HadoopKerberosName.setRuleMechanism(Ljava/lang/String;)V at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerb

org.apache.spark.SparkException: Exception thrown in awaitResult"time":"2020-07-02 11:10:15","logtype":"ERROR","loginfo":"Executor self-exiting due to : Driver masterba17fc94-f00c-477a-b1d4-4c165c7893d9cupid-11-194-251-15:45755 disassociated! Shutting dow

security_exception: missing authentication credentials for REST request [/] nullCaused by: org.elasticsearch.hadoop.rest.EsHadoopInvalidRequest: org.elasticsearch.hadoop.rest.EsHadoopRemoteExceptio...

在部署hadoop时 ,使用shell脚本远程 ssh启动,zk集群启动失败整理:1.报错信息

背景: 单机部署卡夫卡伪分布式 发现仅启动了两台 还有一台 kafka启动不起查看内存内存ok,足够启动[hadoop@hadoop001 config]$ free -h total used free shared buff/cache availableMem: 7.6G 3...

java.lang.NumberFormatException: Not a version: 9SCALA对应的版本不一致导致, flink1.9.1对应scala2.11.82. java.lang.ClassNotFoundException: org.apache.flink.api.scala.typeutils.CaseClassTypeInfo原因 jar引入错误...

Client requested master to start replication from position Client requested master to start replication from positioncom.github.shyiko.mysql.binlog.network.ServerException: Client requested master ...

1. 发现问题:发现CentOS7.3 进程中 rsyslogd cpu的占用率 高达 99%2 解决问题发现rsyslog可以理解为增强版的syslog, 可以支持输出日志到各种数据库, 使用RELP + TCP 实现数据的传输,对目前的服务器服务而言,可以关闭 该进程# 第一步:重启rsyslog 服务,发现 进程cpu 占用率依旧高达99%,采取 第二种方案[root@h...

1. 问题发生场景:window 环境,使用idea 开发Spark作业,并 运行job作业,报错{"time":"2020-01-19 11:24:41","logtype":"WARN","loginfo":"Unable to load native-hadoop library for your platform... using builtin-java classes wher...

19/10/14 16:33:00 ERROR Executor: Exception in task 0.0 in stage 0.0 (TID 0)java.lang.RuntimeException: Error while encoding: java.lang.RuntimeException: scala.Tuple2 is not a valid external type for...

1.Connecting to ResourceManager at /0.0.0.0:803219/09/04 10:54:16 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:803219/09/04 10:54:19 INFO ipc.Client: Retrying connect to server: 0....

1.org.apache.hadoop.security.AccessControlException: Permission denied:产生原因:用户没有相对应的权限修改文件,或执行文件org.apache.hadoop.security.AccessControlException: Permission denied: user=Administrator, access=WR...

java.lang.ClassNotFoundException: org.apache.curator.connection.ConnectionHandlingPolicy出现问题的原因:curator与zookeeper版本不兼容导致java.lang.NoClassDefFoundError: org/apache/curator/connection/ConnectionHand...