部署CDH以及相关服务启动时的问题与解决方法



        部署CDH以及相关服务启动时的问题与解决方法
(一)系统配置问题
  (1)磁盘空间不够
    The I/O cache encountered an error while updating data in medium "ahci-0-0" (rc=VERR_BAD_EXE_FORMAT). Make sure there is enough free space on the disk and that the disk is working properly. Operation can be resumed afterwards.
   解决:
 三台机器的配置
  192.168.1.(164~166):内存4G,磁盘40G


(二)cloudera-scm-server服务
  (1)cloudera-scm-server启动后马上down掉
       # /etc/init.d/cloudera-scm-server status
       cloudera-scm-server 已死,但 pid 文件仍存  
   解决:
     1. cloudera-scm-server依赖Mysqld服务,启动必须先启动数据库服务  
      #/etc/init.d/mysqld start
      #chkconfig mysqld on    (实现开机自启动)
    2. 解析域名vim /etc/hosts 
          IP  名称
  (2)cloudera-scm-server 启动后挂掉
     查看日志    
    # vim /var/log/cloudera-scm-server/cloudera-scm-server.log
ERROR main:com.cloudera.server.cmf.bootstrap.EntityManagerFactoryBean: Tables have unsupported engine type [MyISAM]. InnoDB is required.
     数据库要求INNODB
   解决:
   # vim /etc/my.cnf
    [mysqld]
    default-storage-engine=INNODB
   重启服务
    #/etc/init.d/mysld restart
   重置数据库
    #/usr/share/cmf/schema/scm_prepare_database.sh mysql -h localhost -uroot -p'123456' --scm-host localhost scm scm scm
    #用Cloudera Manager重新建集群


(三)安装CDH
   (1)  yum源报错(安装服务前提前搭建好http服务)
https://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.10.0/repodata/repomd.xml: [Errno 14] PYCURL ERROR 6 - "Couldn't resolve host 'archive.cloudera.com'"
    原因:cloudera-agent的URL不正确
    解决:搭建http服务器,创建的yum源
          
   1. 搭建http服务
      #service httpd start
      #mkdir /var/www/html/parcel-repo/
      #ln -s /opt/cloudera/parcel-repo/  /var/www/html/parcel-repo/
   2.创建parcel的yum源   
     #yum install createrepo  -y
     #cd /var/www/html/parcel-repo/
     #createrepo  .
3.浏览器测试
    http://192.168.3.82/parcel-repo/


解决cloudera Manager安装锁的问题
     删掉/tmp下面的
     #rm -rf  scm_prepare_node.*
     # rm -rf  .scm_prepare_node.lock
 
无法连接HOST_Monitor的问题(安装完CDH)

   (1)确定启动cloudera Managemnt服务

            

   
   (2)agent无法连接HOST_Monitor的问题
        1. 查看日志
         # cat /var/log/cloudera-scm-agent/cloudera-scm-agent.log
      INFO CM server guid: d31b5fae-56f4-466e-9f89-d639daa5038b
        2. 删除cm_guid
          #rm -rf  /var/lib/cloudera-scm-agent/cm_guid 
        3. 重启服务
           #/etc/init.d/cloudera-scm-agent restart
  guid是agent首次启动时会从server处获得,之后一直保存在agent端。Host-Monitor服务是运行在server端。guid是agent连接server端Host-Monitor服务的一个标识


  (3)测某台主机连接不上Host-Monitor
   Error sending messages to firehose: mgmt-SERVICEMONITOR-ddc9b774d2da234ec1da3086c2a9355f
  解决:
     查看是否集群中所有IP与域名对应:vim /etc/hosts
     重启agent: /etc/init.d/cloudera-scm-agent restart


 (4)查询Host-Monitor内部出错
       Could not fetch descriptor after 5 tries, exiting
     解决:重启cloudera Manager服务   


 (5)集群安装报错
      ERROR  Failure due to stall on seeded torrent.
    解决:重启agent服务
      
(五)oozie服务启动报错  
    E0103: Could not load service classes, Cannot load JDBC driver class 'com.mysql.jdbc.Driver'
    解决:
     软连接JDBC的驱动器
    #ln -s /soft/mysql-connector-java-5.1.40-java.jar /opt/cloudera/parcels/CDH-5.10.0-1.cdh5.10.0.p0.41/lib/oozie/lib/


(六)HDFS启动时DataNode异常
   Initialization failed for Block pool <registering> (Datanode Uuid 526147d4-7dbe-4f8a-acc8-46da63ed8828) service to test1/192.168.1.164:8022. Exiting. 
   解决:
    #vim  /dfs/dn/current/VERSION 
      clusterID=cluster6    //保证在同一集群中


(七)Hive启动后Hive Meatstore server进程异常退出
   1)数据库驱动的问题 
       错误1:Error creating transactional connection factory
          cause by:Attempt to invoke the "BONECP" plugin to create a ConnectionPool gave an error : The specified datastore driver ("com.mysql.jdbc.Driver") was not found in the CLASSPATH. Please check your CLASSPATH specification, and the name of the driver.
     解决:
#ln  -s  /soft/mysql-connector-java-5.1.40-bin.jar  /opt/cloudera/parcels/CDH/lib/hive/auxlib


   2)数据库字符问题
      错误2:Specified key was too long; max key length is 767 bytes
      原因:Mysql的utf8字符集最大长度限制,注意字符集设置为latin1
      解决:
        在CM的web UI界面设置Hive的数据库:

            [Hive]-->[配置]-->[Hive Metastore数据库]-->[Hive Metastore数据库名称]


                 
        创建Metastore数据库
        Mysql>mysql -uroot -p123456
        mysql>create database hive character set latin1;


   3)数据库权限问题
      错误3:  
       Database access problem. Killing off this connection and all remaining connections in the connection pool. SQL State = HY000
       解决:
          grant  all on hive.* to 'scm'@'192.168.1.164' identified by 'scm';
          grant  all on hive.* to 'scm'@'192.168.1.165' identified by 'scm';
          grant  all on hive.* to 'scm'@'192.168.1.166' identified by 'scm';


   4)错误4:
     MetaException(message:Version information not found in metastore. )
     解决:   
       在CM的web UI界面配置Hive服务:
               
      在hive配置文件中添加:
      #cd /opt/cloudera/parcels/CDH/lib/hive/conf
       # vim hive-site.xml
         <property>
            <name>hive.metastore.schema.verification</name>
            <value>false</value>
         </property>
评论 2
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值