ambari2.1离线安装HDP2.1实践教程

说明：此文档是讲解ambari离线安装HDP的方法，这里的离线安装说的是ambari组件、HDP组件这些大的文件从网站上下载好避免网络不好中间中断，但是有些小文件如PostgreSQL、系统初始化yum更新、mysql之类的包也需要网络，当然可以本地yum源，只是麻烦些，建议还是需要服务器可以访问外网。

在线安装可参考：http://blog.csdn.net/levy_cui/article/details/59108546 因为线上环境是3年前安装的，所以版本比较低，新安装可以尝试较新的版本，安装步骤是一样的。

AMBARI-2.4.0.1-centos6.tar.gz
包含HDP 2.2 - HDP 2.5版本

ambari-2.1.0-centos6.tar.gz

包含HDP 2.1 - HDP 2.4版本

准备工作：

安装集群前需要进行系统初始化设置，如主机hosts、ssh免秘钥、文件句柄打开数、防火墙selinux关闭等等，网上很多自行搜索。

一、下载：
cd /usr/local/src/
http://s3.amazonaws.com/public-repo-1.hortonworks.com/ambari/centos6/ambari-2.1.0-centos6.tar.gz (482M)
http://s3.amazonaws.com/public-repo-1.hortonworks.com/HDP/centos6/2.x/2.1-latest/HDP-2.1-latest-centos6-rpm.tar.gz (2.6G)
http://s3.amazonaws.com/public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/centos6/HDP-UTILS-1.1.0.20-centos6.tar.gz (694M)

当这些文件下载后，将这些压缩包的文件解压到 /var/www/html中，为了方便管理建议在该目录下创建一个external目录

使用tar命令解压缩：
yum install httpd
mkdir -p /var/www/html/external //新建目录，将下载的文件解压到/var/www/html/external中
tar -xvf /usr/local/src/HDP-2.1-latest-centos6-rpm.tar.gz -C /var/www/html/external/
tar -xvf /usr/local/src/HDP-UTILS-1.1.0.20-centos6.tar.gz -C /var/www/html/external/
tar -xvf /usr/local/src/ambari-2.1.0-centos6.tar.gz -C /var/www/html/external/

之后，在/etc/yum.repos.d创建三个repo文件

#vi /etc/yum.repos.d/ambari.repo
[Updates-ambari-2.1.0]
name=ambari-2.1.0 - Updates
baseurl=http://hadooptest76.bj/external/ambari-2.1.0/centos6
gpgcheck=1
gpgkey=http://hadooptest76.bj/external/ambari-2.1.0/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1

#vi /etc/yum.repos.d/hdp.repo
[HDP-2.1.10.0]
name=HDP Version - HDP-2.1.10.0
baseurl=http://hadooptest76.bj/external/HDP/centos6/2.x/updates/2.1.10.0
gpgcheck=1
gpgkey=http://hadooptest76.bj/external/HDP/centos6/2.x/updates/2.1.10.0/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1

[HDP-UTILS-1.1.0.20]
name=HDP Utils Version - HDP-UTILS-1.1.0.20
baseurl=http://hadooptest76.bj/external/HDP-UTILS-1.1.0.20/repos/centos6
gpgcheck=1
gpgkey=http://hadooptest76.bj/external/HDP/centos6/2.x/updates/2.1.10.0/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1

#vi /etc/yum.repos.d/hdp-util.repo
[HDP-UTILS-1.1.0.20]
name=Hortonworks Data Platform Version - HDP-UTILS-1.1.0.20
baseurl= http://hadooptest76.bj/external/HDP-UTILS-1.1.0.20/repos/centos6
gpgcheck=1
enabled=1
priority=1

之后，将写好的文件，发送至其他节点上（这里只举一个例子，需要scp到所有节点上）
scp ambari.repo root@Test02:/etc/yum.repo.d/
scp hdp.repo root@Test02:/etc/yum.repo.d/
scp hdp-util.repo root@Test02:/etc/yum.repo.d/

二、在ambari服务节点运行
yum install ambari-server

下载或者从其他节点scp jdk（我这里之前已经下载过了所有直接scp一份到ambari节点上）
scp /var/lib/ambari-server/resources/jdk-7u67-linux-x64.tar.gz 10.11.32.76:/var/lib/ambari-server/resources/


三、安装MySQL服务
1）安装
通过yum安装mysql：
[root@cluster-02 ~]$ yum install mysql-server
启动mysql服务
[root@cluster-02 ~]$ service mysqld start

安装MySQL JDBC Connector：
[root@cluster-02 root]$ yum install mysql-connector-java

2）为root用户设置新密码等初始化工作
[root@cluster-01 ~]$ /usr/bin/mysql_secure_installation

3）为ambari创建数据库，配置相应用户和权限
[root@cluster-02 ~]# mysql -u root -p

mysql> create database ambari;
Query OK, 1 row affected (0.00 sec)

mysql> use ambari;
Database changed

4)hive库
mysql> create database hive;
mysql> create user "hive" identified by "hive123";
mysql> grant all privileges on hive.* to hive;
mysql> flush privileges;

5）启动
mysql设置为开机自启动
[root@cluster-05 root]$ chkconfig mysqld on
[root@cluster-05 root]$ chkconfig --list
mysqld             0:off      1:off      2:on      3:on      4:on      5:on      6:off

6)导入数据库
注意，此刻需要切换到mysql中执行相应脚本操作。
需要登录到mysql数据库，执行以下脚本Ambari-DDL-MySQL-CREATE.sql
[root@ambari ~]# mysql -u root -p
mysql> use ambari;
Database changed
mysql> source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
执行脚本，成功。

ambari权限设置
CREATE USER 'ambari'@'%' IDENTIFIED BY 'bigdata';
GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'%';
FLUSH PRIVILEGES;


四、ambari server安装
[root@Ambari yum.repos.d]# ambari-server setup
Using python  /usr/bin/python2.6
Setup ambari-server
Checking SELinux...
SELinux status is 'disabled'
Customize user account for ambari-server daemon [y/n] (n)? （直接回车）
Adjusting ambari-server permissions and ownership...
Checking firewall...
WARNING: iptables is running. Confirm the necessary Ambari ports are accessible. Refer to the Ambari documentation for more details on ports.
OK to continue [y/n] (y)? （输入y直接回车）
Checking JDK...
[1] - Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7
[2] - Oracle JDK 1.6 + Java Cryptography Extension (JCE) Policy Files 6
[3] - Custom JDK
==============================================================================
Enter choice (1):1 （直选择1 根据版本选择）
To download the Oracle JDK and the Java Cryptography Extension (JCE) Policy Files you must accept the license terms found at

http://www.oracle.com/technetwork/java/javase/terms/license/index.html and not accepting will cancel the Ambari Server setup and you must install the JDK and JCE files

manually.
Do you accept the Oracle Binary Code License Agreement [y/n] (y)? （直接回车）
Downloading JDK from http://public-repo-1.hortonworks.com/ARTIFACTS/jdk-7u67-linux-x64.tar.gz to /var/lib/ambari-server/resources/jdk-7u67-linux-x64.tar.gz
JDK distribution size is 142376665 bytes
jdk-7u67-linux-x64.tar.gz... 100% (135.8 MB of 135.8 MB)
Successfully downloaded JDK distribution to /var/lib/ambari-server/resources/jdk-7u67-linux-x64.tar.gz
Installing JDK to /usr/jdk64
Successfully installed JDK to /usr/jdk64/jdk1.7.0_67
Downloading JCE Policy archive from http://public-repo-1.hortonworks.com/ARTIFACTS/UnlimitedJCEPolicyJDK7.zip to /var/lib/ambari-

server/resources/UnlimitedJCEPolicyJDK7.zip
Successfully downloaded JCE Policy archive to /var/lib/ambari-server/resources/UnlimitedJCEPolicyJDK7.zip
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)? y （y回车）
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL
[4] - PostgreSQL
==============================================================================
Enter choice (1): 3
Hostname (localhost): （直接回车）
Port (3306): （直接回车）
Database Name (ambari): （直接回车）
Username (ambari): （直接回车）
Enter Database Password (bigdata): （输入ambari密码，我这里是默认bigdata）
Copying JDBC drivers to server resources...
Configuring remote database connection properties...
WARNING: Before starting Ambari Server, you must run the following DDL against the database to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-

CREATE.sql
Proceed with configuring remote database connection properties [y/n] (y)? （直接回车）
Copying JDBC drivers to server resources...
Extracting system views...
ambari-admin-1.7.0.169.jar
..
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.


[root@ambari ~]# ambari-server start
Using python  /usr/bin/python2.6
Starting ambari-server
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start....................
Ambari Server 'start' completed successfully.

使用浏览器打开IP:8080, 输入admin/admin 即可访问ambari管理界面开始HDP集群安装。

五、安装HDP（包含hadoop、zookeeper、hive等等）

离线需要在界面中源路径配置相关内容：

Use Local Repository

http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.2.6.0
http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/centos6

修改为：
http://hadooptest76.bj/external/HDP/centos6/2.x/updates/2.1.10.0/
http://hadooptest76.bj/external/HDP-UTILS-1.1.0.20/repos/centos6


配置hive的时候需要安装mysql-connector-java.jar
find / -name mysql-connector-java.jar
ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java.jar

其余内容可参考http://www.cnblogs.com/tgzhu/tag/HDP/ 写的挺详细了

参考：
http://www.cnblogs.com/roger888/p/5505731.html
http://blog.csdn.net/hyd38888/article/details/48136461
http://blog.csdn.net/poisonchry/article/details/37726663