大数据
justinzwd
脚踏实地
展开
-
kafka自动宕机问题解决
最近在做Spark和Flink项目的时候,总是发现Kafka在启动成功之后一小段时候就自动宕机通过查询日志发现Connecting to localhost:21812021-02-12 01:20:56,083 [myid:] - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017原创 2021-02-12 16:41:46 · 1219 阅读 · 0 评论 -
解决报错 YumRepo Error: All mirror URLs are not using ftp, http[s] or file.
问题CentOS 系统中出现YumRepo Error: All mirror URLs are not using ftp, http[s] or file.报错Loaded plugins: fastestmirror, refresh-packagekit, securitySetting up Update ProcessDetermining fastest mirrorsYumRepo Error: All mirror URLs are not using ftp, http[s]原创 2021-02-10 11:09:34 · 11115 阅读 · 13 评论 -
启动spark shell 报错
启动 bin/spark-shell的时候出现如下报错信息Using Spark's default log4j profile: org/apache/spark/log4j-defaults.propertiesSetting default log level to "WARN".To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).21/01/19 10:45:5原创 2021-01-19 10:05:28 · 945 阅读 · 0 评论 -
mapreduce中的context类
在Mapper中的map、以及Reducer中的reduce都有一个Context的类型1 public void map(Object key, Textvalue, Context context)2 throwsOException,InterruptedException{3 StringTokenizer itr = new StringTok...转载 2019-05-06 20:47:59 · 1822 阅读 · 0 评论 -
load加载本地数据到hive数据库后查询数据为NULL
hive> show databases;OKdefaultTime taken: 1.867 seconds, Fetched: 1 row(s)hive> use default;OKTime taken: 0.067 secondshive> show tables;OKstudentTime taken: 0.072 seconds, Fetched:...原创 2019-05-11 16:29:10 · 2861 阅读 · 1 评论