![](https://img-blog.csdnimg.cn/20201014180756919.png?x-oss-process=image/resize,m_fixed,h_64,w_64)
bugbug
binbincoder
生而无畏、战至终章
展开
-
Centos7.5(64)安装Oracle 11g2问题汇总
环境说明:系统环境:CentOS7.5 64位JDK版本:JDK8Oracle版本:11g2问题一:初始安装时检查交换空间: 可用的交换空间为 0 MB, 所需的交换空间为 150 MB。 未通过(root用户下操作)The available swap space is 0 MB, and the required swap space is 150 MB. Not through问题解决:$ sudo fallocate -l 512m /mnt/512MiB.swap$ sudo ch原创 2020-11-05 11:42:30 · 780 阅读 · 0 评论 -
windows安装Flume将本地目录下文件上传到HDFS(kerberos)问题汇总
环境本地系统:Windows10CDH:6.1.0Hadoop: 3.0Flume:1.8.0JDK:1.8安全:kerberos问题一:找不到com/ctc/wstx/io/InputBootstrapper2020-09-02 11:02:22,640 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSin原创 2020-09-04 09:59:35 · 838 阅读 · 0 评论 -
Spark(本地)操作Hadoop常见问题汇总
问题一: Failed to connect to bioyuana/ip:1004 for block BP-1366618129-172.16.1.176-1576652184324:blk_1073760623_19803, add to deadNodes and continue.20/05/13 11:40:32 WARN DFSClient: Failed to connect to bioyuana/ip:1004 for block BP-1366618129-172.16.1.176-原创 2020-05-13 12:00:58 · 561 阅读 · 0 评论 -
StreamSets从MySQL实时同步数据到Hbase问题总结
问题一:找不到zookeeper的/hbase/masterorg.apache.hadoop.hbase.MasterNotRunningException: org.apache.hadoop.hbase.MasterNotRunningException: java.io.IOException: org.apache.zookeeper.KeeperException$Connectio...原创 2020-03-17 16:03:00 · 643 阅读 · 0 评论 -
Docker Python启动报:Bind for 0.0.0.0:8080 failed: port is already allocated
一句话总结就是容器占用的port还没有完全释放查看进程,发现相关的容器并没有在运行,而 docker-proxy 却依然绑定着端口:$ docker ps检查docker镜像$ ps -aux | grep -v grep | grep docker-proxy查看被docker-proxy占用的端口停止 doker 进程,删除所有容器,然后删除 local-kv.db 这个文件,...原创 2019-09-20 09:52:26 · 4056 阅读 · 0 评论 -
Error creating LevelDB timeseries store in directory /var/lib/cloudera-service-monitor/ts
错误描述:重启CLoudera Manager集群的Cloudera Management Service报如下错误!!!错误问题:上午10点14:04.184分 ERROR Main Error creating LevelDB timeseries store in directory /var/lib/cloudera-service-monitor/ts java.io.IOEx...原创 2019-06-27 10:24:08 · 2129 阅读 · 0 评论 -
Access denied for user 'root'@'bioyuana' to database 'oozie'
FATAL Services SERVER[bioyuanA] E0103: Could not load service classes, Cannot create PoolableConnectionFactory (Access denied for user 'root'@'bioyuana' to database 'oozie')org.apache.oozie.service....原创 2019-05-06 15:00:27 · 282 阅读 · 0 评论 -
Permission denied: user=hbase, access=WRITE, inode="/":root:supergroup:drwxr-xr-x
FATAL HMaster Failed to become active masterorg.apache.hadoop.security.AccessControlException: Permission denied: user=hbase, access=WRITE, inode="/":root:supergroup:drwxr-xr-x at org.apache.hadoop...原创 2019-05-06 11:50:09 · 2332 阅读 · 0 评论 -
java.io.FileNotFoundException: /opt/dfs/nn/current/VERSION (Permission denied)
ERROR NameNode Failed to start namenode.java.io.FileNotFoundException: /opt/dfs/nn/current/VERSION (Permission denied)at java.io.RandomAccessFile.open0(Native Method)at java.io.RandomAccessFile.op...原创 2019-05-06 11:37:20 · 2778 阅读 · 0 评论 -
elasticSearch安装了x-pack之后无法连接head问题
描述:启动了es之后可以访问:node1:9200,也可以访问node3:9100,但是head的可视化页面无法连接到elasticSearch集群.问题解决:方案一:在elasticsearch.yml里加这么一行xpack.security.enabled: false方案二:在elasticsearch.yml里加这么两行http.cors.enabled: truehtt...原创 2018-12-25 15:20:08 · 2478 阅读 · 0 评论 -
集群搭建准备工作
1)20台干净的虚拟机 2)静态IP已经配置 vi /etc/udev/rules.d/70-persisitent-net.rules vi /etc/sysconfig/network-sc/ifcfg-eth0 static hwaddr ipaddr...原创 2018-11-27 19:44:26 · 160 阅读 · 1 评论 -
RHSA-2018:3157-中危: curl 和 nss-pem 安全和BUG修复更新
Curl低版本重定向中的HTTP身份验证泄漏漏洞:官网解释:https://curl.haxx.se/docs/CVE-2018-1000007.htmlCurl版本升级解决漏洞:https://www.cnblogs.com/kingsonfu/p/10069755.html防止原地址不存在:1.安装reporpm -Uvh http://www.city-fan.org/f...原创 2020-01-13 16:57:05 · 771 阅读 · 0 评论 -
MapReduce中文乱码
问题:mapreduce输出乱码原因:再用Hadoop处理数据的时候,发现输出的时候,总是会出现乱码,这是因为Hadoop在设计编码的时候,是写死的。默认是UTF-8,所以当你处理的文件编码格式不是为UTF-8的时候,比如为GBK格式,那么就会输出的时候就会出现乱码。问题解决:解决问题非常简单,就是转码,确定数据都是以UTF-8的编码格式在运行。在map端从文件中读取一行数据的时候,把...原创 2018-11-28 19:22:29 · 1175 阅读 · 0 评论 -
Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org/apache/hadoop/hbase/
**hive表关联HBase时候遇到的错误建表语句如下:** create external table ext_calllogs_in_hbase(id string, caller string,callee string,time string,dur string) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' ...原创 2018-12-08 19:38:07 · 1031 阅读 · 0 评论 -
无法联系到 HBase Thrift 1 服务器:Could not connect to localhost:9090
**错误叙述**: 集成HBase与Hue是报如下错误: 无法联系到 HBase Thrift 1 服务器:Could not connect to localhost:9090错误原因:因为hbase与Hue集成时需要开启9095端口.就是thrift解决方案:...原创 2018-12-03 20:04:41 · 1822 阅读 · 0 评论 -
{"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication token for REST
错误如下:{"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication token for REST request [/idx]","header":{"WWW-Authenticate":"Basic realm=\&quo原创 2018-12-13 17:14:23 · 35409 阅读 · 9 评论 -
java.lang.RuntimeException: java.io.EOFException
错误问题:ava.lang.RuntimeException: java.io.EOFExceptionat org.apache.hadoop.io.WritableComparator.compare(WritableComparator.java:165)at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.compare(MapTas...原创 2018-12-11 11:00:40 · 3459 阅读 · 1 评论 -
ImportError: cannot import name 'multiarray' from 'numpy.core'
Python加载numpy报错如下:D:\LightGBM\LightGBM\examples\python-guide>python simple_example.pyTraceback (most recent call last):File “simple_example.py”, line 4, in import lightgbm as lgbFile “D:\Pytho...原创 2018-12-29 10:36:06 · 5260 阅读 · 0 评论 -
Caused by: org.elasticsearch.bootstrap.BootstrapException: java.nio.file.NoSuchFileException: /home/
问题:elasticSearch安装Ik的坑启动时报错如下:Caused by: org.elasticsearch.bootstrap.BootstrapException: java.nio.file.NoSuchFileException: /home/hadoop/install/elasticsearch-5.6.1/plugins/elasticsearch/plugin-desc...原创 2018-12-25 16:05:52 · 5736 阅读 · 3 评论 -
java.lang.NoSuchMethodError: org.apache.hadoop.mapred.TaskID.<init>
java.lang.NoSuchMethodError: org.apache.hadoop.mapred.TaskID.(Lorg/apache/hadoop/mapreduce/JobID;Lorg/apache/hadoop/mapreduce/TaskType;I)Vat org.apache.spark.rdd.HadoopRDD.addLocalConfiguration(Hadoo...原创 2019-01-24 10:12:53 · 1647 阅读 · 2 评论 -
java.net.ConnectException: Connection timed out: no further information
首先,本地读取云集群hdfs文件报连接超时,这个问题困的我。。。。。。。百度,谷歌很多都是防火墙,端口,路径的错误,这些都不是这个问题的根源,防火墙开着也可以读取hdfs中的文件,所以有必要记下这个问题,错误如下:19/06/03 15:25:09 WARN BlockReaderFactory: I/O error constructing remote block reader. jav...原创 2019-06-03 15:37:20 · 6616 阅读 · 3 评论 -
Kafka读取数据是中文乱码问题
解决方案一解决方案二redis 中文存储乱码问题在redis 中存储中文,读取会出现乱码(其实不是乱码,只是不是我们存的中文显示)1 redis> set test “我们”2 OK3 redis> get test4 “\xe6\x88\x91\xe4\xbb\xac”如何在get时取到它的中文呢?只需要在redis-cli 后面加上 --rawredis-cli...原创 2018-11-23 09:51:30 · 19705 阅读 · 1 评论