Ubuntu14 4 安装 Cloudera Manager5及CDH5(Mysql)

以三个节点为例，搭建cloudera集群。

总体思路是，现在master1上安装好cloudera，然后在cloudera manager上通过添加hosts的方式加入其他两个节点。

1./etc/hosts设置

按master1,slave1,slave2给三台机器配置hostname

修改所有节点的/etc/hosts

1. 192.168.23.131 master1
   192.168.23.130 slave1
   192.168.23.132 slave2
   

确保所有机器root密码一致或者sudo免输入密码用户的用户名、密码一致！

2.添加Cloudera源

1.添加源

1. sudo nano /etc/apt/sources.list.d/cloudera.list

写入

1. # Packages for Cloudera Manager, Version 5, on Ubuntu 14.04 x86_64
   deb [arch=amd64] http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm trusty-cm5 contrib
   deb-src http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm trusty-cm5 contrib
   # Packages for Cloudera's Distribution for Hadoop, Version 5, on Ubuntu 14.04 amd64       
   deb [arch=amd64] http://archive.cloudera.com/cdh5/ubuntu/trusty/amd64/cdh trusty-cdh5 contrib
   deb-src http://archive.cloudera.com/cdh5/ubuntu/trusty/amd64/cdh trusty-cdh5 contrib
   

http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm/ 为cm的源

http://archive.cloudera.com/cdh5/ubuntu/trusty/amd64/cdh trusty-cdh5 为chd5的源

上述源都是对应的ubuntu14的源，如果需要找到ubuntu10和ubuntu12的源，请在

对应目录寻找cloudera.list

lucid -> ubuntu10 precise -> ubuntu12 trusty -> ubuntu14

获取apt key

1. curl -s http://http://archive.cloudera.com/cdh5/ubuntu/trusty/amd64/cdh/archive.key| sudo apt-key add -

刷新源列表

1. sudo apt-get update

3.安装java环境

1. sudo apt-get -o Dpkg::Options::=--force-confdef -o Dpkg::Options::=--force-confold -y install oracle-j2sdk1.7

java的安装路径参考：  /usr/lib/jvm

4.安装mysql以及JDBC驱动

1. sudo apt-get install mysql-server libmysql-java

5.配置mysql

1. sudo nano /etc/mysql/conf.d/mysql_cloudera_manager.cnf
   

复制代码

写入

1. [mysqld]
   
2. transaction-isolation=READ-COMMITTED
   
3. # Disabling symbolic-links is recommended to prevent assorted security risks;
   
4. # to do so, uncomment this line:
   
5. # symbolic-links=0
   
6. 
   
7. key_buffer              = 16M
   
8. key_buffer_size         = 32M
   
9. max_allowed_packet      = 16M
   
10. thread_stack            = 256K
    
11. thread_cache_size       = 64
    
12. query_cache_limit       = 8M
    
13. query_cache_size        = 64M
    
14. query_cache_type        = 1
    
15. # Important: see Configuring the Databases and Setting max_connections
    
16. max_connections         = 550
    
17. 
    
18. # log-bin should be on a disk with enough free space
    
19. log-bin=/var/log/mysql/mysql_binary_log
    
20. 
    
21. # For MySQL version 5.1.8 or later. Comment out binlog_format for older versions.
    
22. binlog_format           = mixed
    
23. 
    
24. read_buffer_size = 2M
    
25. read_rnd_buffer_size = 16M
    
26. sort_buffer_size = 8M
    
27. join_buffer_size = 8M
    
28. 
    
29. # InnoDB settings
    
30. innodb_file_per_table = 1
    
31. innodb_flush_log_at_trx_commit  = 2
    
32. innodb_log_buffer_size          = 64M
    
33. innodb_buffer_pool_size         = 4G
    
34. innodb_thread_concurrency       = 8
    
35. innodb_flush_method             = O_DIRECT
    
36. innodb_log_file_size = 512M
    
37. 
    
38. [mysqld_safe]
    
39. log-error=/var/log/mysqld.log
    
40. pid-file=/var/run/mysqld/mysqld.pid

复制代码

在安装的过程中一定保证内存足够大，否则会遇到下面问题
上面配置需要根据自己的实际情况，在配置过程中重启mysql的时候，发生了下面错误

stop: Unknown instance: 
start: Job failed to start

注意：不能够重启是因为这两项的原因
innodb_buffer_pool_size = 4G
log-bin=/var/log/mysql/mysql_binary_log





编辑my.cnf
sudo nano /etc/mysql/my.cnf
把下面这一行注释掉

1. #bind-address           = 127.0.0.1

6.配置innodb

1. sudo su
   
2. mv /var/lib/mysql/ib_logfile* /var/tmp/
   
3. logout

7.初始化数据库

1. sudo service mysql restart

1. mysql -uroot -p

1. create database amon DEFAULT CHARACTER SET utf8;
   
2. grant all on amon.* TO 'amon'@'%' IDENTIFIED BY 'amon_password';
   
3. grant all on amon.* TO 'amon'@'CDH' IDENTIFIED BY 'amon_password';
   
4. create database smon DEFAULT CHARACTER SET utf8;
   
5. grant all on smon.* TO 'smon'@'%' IDENTIFIED BY 'smon_password';
   
6. grant all on smon.* TO 'smon'@'CDH' IDENTIFIED BY 'smon_password';
   
7. create database rman DEFAULT CHARACTER SET utf8;
   
8. grant all on rman.* TO 'rman'@'%' IDENTIFIED BY 'rman_password';
   
9. grant all on rman.* TO 'rman'@'CDH' IDENTIFIED BY 'rman_password';
   
10. create database hmon DEFAULT CHARACTER SET utf8;
    
11. grant all on hmon.* TO 'hmon'@'%' IDENTIFIED BY 'hmon_password';
    
12. grant all on hmon.* TO 'hmon'@'CDH' IDENTIFIED BY 'hmon_password';
    
13. create database hive DEFAULT CHARACTER SET utf8;
    
14. grant all on hive.* TO 'hive'@'%' IDENTIFIED BY 'hive_password';
    
15. grant all on hive.* TO 'hive'@'CDH' IDENTIFIED BY 'hive_password';

8.安装 cloudera-manager以及agent(因为master也是一个节点)

1. sudo apt-get install cloudera-manager-daemons cloudera-manager-server  cloudera-manager-agent

9.配置cloudera-manager-server数据库

1. sudo /usr/share/cmf/schema/scm_prepare_database.sh mysql  -uroot -p --scm-host localhost scm scm scm_password

这里执行的时候，如果设置了mysql的密码，那么需要输入mysql密码，并且会出现log4j错误，不用担心。