ubuntu 12.04 安装 Cloudera Manager 4.8.2

Cloudera可以简化Hadoop的安装配置过程，自动在集群节点上安装hadoop相关组件，创建用户。Cloudera是Hadoop生态系统中，规模最大、知名度最高的公司

1.准备

系统版本

1	iflab@cdh1:~$ uname -a

2	Linux cdh1 3.11.0-15-generic #25~precise1-Ubuntu SMP Thu Jan 30 17:39:31 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

3	iflab@cdh1:~$

角色分配(需要修改hostname、hosts)

1	IP               主机名

2	222.249.250.211  cdh1    manager master

3	222.249.250.212  cdh2    slave1

4	222.249.250.213  cdh3    slave2

5	222.249.250.214  cdh4    slave3

确保所有机器root密码一致或者sudo免输入密码用户的用户名、密码一致！

2.添加Cloudera源

1	sudo sudo nano /etc/apt/sources.list.d/cloudera.list

写入

1	deb [arch=amd64] http://archive.cloudera.com/cdh4/ubuntu/precise/amd64/cdh precise-cdh4 contrib

2	deb-src http://archive.cloudera.com/cdh4/ubuntu/precise/amd64/cdh precise-cdh4 contrib

3	deb [arch=amd64] http://archive.cloudera.com/cm4/ubuntu/precise/amd64/cm precise-cm4 contrib

4	deb-src http://archive.cloudera.com/cm4/ubuntu/precise/amd64/cm precise-cm4 contrib

获取apt key

1	curl -s http://archive.cloudera.com/cdh4/ubuntu/precise/amd64/cdh/archive.key | sudo apt-key add -

刷新源列表

1	sudo apt-get update

3.安装java环境

1	sudo apt-get -o Dpkg::Options::=--force-confdef -o Dpkg::Options::=--force-confold -y install oracle-j2sdk1.6

4.安装mysql以及JDBC驱动

1	sudo apt-get install mysql-server libmysql-java

5.配置mysql

1	sudo nano /etc/mysql/conf.d/mysql_cloudera_manager.cnf

写入

1

[mysqld]

2	transaction-isolation=READ-COMMITTED

3	# Disabling symbolic-links is recommended to prevent assorted security risks;

4	# to do so, uncomment this line:

5	# symbolic-links=0

6

7	key_buffer              = 16M

8	key_buffer_size         = 32M

9	max_allowed_packet      = 16M

10	thread_stack            = 256K

11	thread_cache_size       = 64

12	query_cache_limit       = 8M

13	query_cache_size        = 64M

14	query_cache_type        = 1

15	# Important: see Configuring the Databases and Setting max_connections

16	max_connections         = 550

17

18	# log-bin should be on a disk with enough free space

19	log-bin=/var/log/mysql/mysql_binary_log

20

21	# For MySQL version 5.1.8 or later. Comment out binlog_format for older versions.

22	binlog_format           = mixed

23

24	read_buffer_size = 2M

25	read_rnd_buffer_size = 16M

26	sort_buffer_size = 8M

27	join_buffer_size = 8M

28

29	# InnoDB settings

30	innodb_file_per_table = 1

31	innodb_flush_log_at_trx_commit  = 2

32	innodb_log_buffer_size          = 64M

33	innodb_buffer_pool_size         = 4G

34	innodb_thread_concurrency       = 8

35	innodb_flush_method             = O_DIRECT

36	innodb_log_file_size = 512M

37

38	[mysqld_safe]

39	log-error=/var/log/mysqld.log

40	pid-file=/var/run/mysqld/mysqld.pid

编辑my.cnf
sudo nano /etc/mysql/my.cnf
把下面这一行注释掉

1	#bind-address           = 127.0.0.1

6.配置innodb

1

sudo su

2	mv /var/lib/mysql/ib_logfile* /var/tmp/

3

logout

7.初始化数据库

1	sudo service mysql restart

2	mysql -uroot -p

3

4	create database amon DEFAULT CHARACTER SET utf8;

5	grant all on amon.* TO 'amon'@'%' IDENTIFIED BY 'amon_password';

6	grant all on amon.* TO 'amon'@'cdh1' IDENTIFIED BY 'amon_password';

7	create database smon DEFAULT CHARACTER SET utf8;

8	grant all on smon.* TO 'smon'@'%' IDENTIFIED BY 'smon_password';

9	grant all on smon.* TO 'smon'@'cdh1' IDENTIFIED BY 'smon_password';

10	create database rman DEFAULT CHARACTER SET utf8;

11	grant all on rman.* TO 'rman'@'%' IDENTIFIED BY 'rman_password';

12	grant all on rman.* TO 'rman'@'cdh1' IDENTIFIED BY 'rman_password';

13	create database hmon DEFAULT CHARACTER SET utf8;

14	grant all on hmon.* TO 'hmon'@'%' IDENTIFIED BY 'hmon_password';

15	grant all on hmon.* TO 'hmon'@'cdh1' IDENTIFIED BY 'hmon_password';

16	create database hive DEFAULT CHARACTER SET utf8;

17	grant all on hive.* TO 'hive'@'%' IDENTIFIED BY 'hive_password';

18	grant all on hive.* TO 'hive'@'cdh1' IDENTIFIED BY 'hive_password';

8.安装 cloudera-manager以及agent(因为master也是一个节点)

1	sudo apt-get install cloudera-manager-daemons cloudera-manager-server  cloudera-manager-agent

9.配置cloudera-manager-server数据库

1	sudo /usr/share/cmf/schema/scm_prepare_database.sh mysql  -uroot -p --scm-host localhost scm scm scm_password

10.配置其余节点（cdh2、cdh3、cdh4）

配置其余节点的源

1	sudo scp iflab@222.249.250.211:/etc/apt/sources.list.d/cloudera.list /etc/apt/sources.list.d/cloudera.list

获取apt key

1	curl -s http://archive.cloudera.com/cdh4/ubuntu/precise/amd64/cdh/archive.key | sudo apt-key add -

刷新源列表

1	sudo apt-get update

在其余节点安装jdk

view source

print ?

1	sudo apt-get -o Dpkg::Options::=--force-confdef -o Dpkg::Options::=--force-confold -y install oracle-j2sdk1.6

在其余节点安装cloudera-manager-agent 和cloudera-manager-daemons

1	sudo apt-get install cloudera-manager-agent cloudera-manager-daemons

修改agent的配置文件
sudo nano /etc/cloudera-scm-agent/config.ini
修改server_host＝cdh1

重启每个节点的agent

1	sudo service cloudera-scm-agent  restart

11.重启控制节点cloudera-manager以及agent

1	sudo service cloudera-scm-server restart

2	sudo service cloudera-scm-agent  restart

12.访问web控制台

http://222.249.250.211:7180/ 用户名admin密码admin