部署CDH以及相关服务启动时的问题与解决方法

（一）系统配置

(1)磁盘空间不够 
     The I/O cache encountered an error while updating data in medium "ahci-0-0" (rc=VERR_BAD_EXE_FORMAT). Make sure there is enough free space on the disk and that the disk is working properly. Operation can be resumed afterwards. 
   解决: 
 三台机器的配置 
  192.168.1.(164~166):内存4G,磁盘40G 


(二)cloudera-scm-server服务 
  (1)cloudera-scm-server启动后马上down掉 
       # /etc/init.d/cloudera-scm-server status 
       cloudera-scm-server 已死,但 pid 文件仍存   
   解决: 
     1. cloudera-scm-server依赖Mysqld服务,启动必须先启动数据库服务   
      #/etc/init.d/mysqld start 
      #chkconfig mysqld on    (实现开机自启动) 
    2. 解析域名vim /etc/hosts  
          IP  名称 
  (2)cloudera-scm-server 启动后挂掉 
     查看日志     
    # vim /var/log/cloudera-scm-server/cloudera-scm-server.log 
 ERROR main:com.cloudera.server.cmf.bootstrap.EntityManagerFactoryBean: Tables have unsupported engine type [MyISAM]. InnoDB is required. 
     数据库要求INNODB 
   解决: 
   # vim /etc/my.cnf 
    [mysqld] 
    default-storage-engine=INNODB 
   重启服务 
    #/etc/init.d/mysld restart 
   重置数据库 
    #/usr/share/cmf/schema/scm_prepare_database.sh mysql -h localhost -uroot -p'123456' --scm-host localhost scm scm scm 
    #用Cloudera Manager重新建集群 

(三)安装CDH 

无法连接HOST_Monitor的问题(安装完CDH) 

   (1)确定启动cloudera Managemnt服务

(2)agent无法连接HOST_Monitor的问题 
        1. 查看日志 
         # cat /var/log/cloudera-scm-agent/cloudera-scm-agent.log 
      INFO CM server guid: d31b5fae-56f4-466e-9f89-d639daa5038b 
        2. 删除cm_guid 
          #rm -rf  /var/lib/cloudera-scm-agent/cm_guid  
        3. 重启服务 
           #/etc/init.d/cloudera-scm-agent restart 
   guid是agent首次启动时会从server处获得,之后一直保存在agent端。Host-Monitor服务是运行在server端。guid是agent连接server端Host-Monitor服务的一个标识 


  (3)测某台主机连接不上Host-Monitor 
   Error sending messages to firehose: mgmt-SERVICEMONITOR-ddc9b774d2da234ec1da3086c2a9355f 
  解决: 
     查看是否集群中所有IP与域名对应:vim /etc/hosts 
     重启agent: /etc/init.d/cloudera-scm-agent restart 


 (4)查询Host-Monitor内部出错 
       Could not fetch descriptor after 5 tries, exiting 
     解决:重启cloudera Manager服务    
(5)集群安装报错 
      ERROR  Failure due to stall on seeded torrent. 
    解决:重启agent服务 
       
(五)oozie服务启动报错   
    E0103: Could not load service classes, Cannot load JDBC driver class 'com.mysql.jdbc.Driver' 
    解决: 
     软连接JDBC的驱动器 
    #ln -s /soft/mysql-connector-java-5.1.40-java.jar /opt/cloudera/parcels/CDH-5.10.0-1.cdh5.10.0.p0.41/lib/oozie/lib/ 


(六)HDFS启动时DataNode异常 
    Initialization failed for Block pool <registering> (Datanode Uuid 526147d4-7dbe-4f8a-acc8-46da63ed8828) service to test1/192.168.1.164:8022. Exiting.  
   解决: 
    #vim  /dfs/dn/current/VERSION  
      clusterID=cluster6    //保证在同一集群中 


(七)Hive启动后Hive Meatstore server进程异常退出 
   1)数据库驱动的问题  
       错误1:Error creating transactional connection factory 
           cause by:Attempt to invoke the "BONECP" plugin to create a ConnectionPool gave an error : The specified datastore driver ("com.mysql.jdbc.Driver") was not found in the CLASSPATH. Please check your CLASSPATH specification, and the name of the driver. 
     解决: 
#ln  -s  /soft/mysql-connector-java-5.1.40-bin.jar  /opt/cloudera/parcels/CDH/lib/hive/auxlib 


   2)数据库字符问题 
      错误2:Specified key was too long; max key length is 767 bytes 
      原因:Mysql的utf8字符集最大长度限制,注意字符集设置为latin1 
      解决: 
        在CM的web UI界面设置Hive的数据库: 

            [Hive]-->[配置]-->[Hive Metastore数据库]-->[Hive Metastore数据库名称]

                  
        创建Metastore数据库 
        Mysql>mysql -uroot -p123456 
        mysql>create database hive character set latin1; 


   3)数据库权限问题 
      错误3:   
       Database access problem. Killing off this connection and all remaining connections in the connection pool. SQL State = HY000 
       解决: 
          grant  all on hive.* to 'scm'@'192.168.1.164' identified by 'scm'; 
          grant  all on hive.* to 'scm'@'192.168.1.165' identified by 'scm'; 
          grant  all on hive.* to 'scm'@'192.168.1.166' identified by 'scm'; 


   4)错误4: 
     MetaException(message:Version information not found in metastore. ) 
     解决:    
       在CM的web UI界面配置Hive服务: 
                 
      在hive配置文件中添加: 
      #cd /opt/cloudera/parcels/CDH/lib/hive/conf 
       # vim hive-site.xml 
         <property> 
            <name>hive.metastore.schema.verification</name> 
            <value>false</value> 
          </property>