Ambari1.7.0 + HDP2.1安装记录

一、准备工作
免秘钥
修改hostname、hosts文件
NTP设置
停止iptabels、selinux
停止THP
修改limit限制
yum install -y openssl bash lrzsz

（以上内容请自行搜索）

二、下载yum源
cd /etc/yum.repos.d/
wget -nv http://public-repo-1.hortonworks.com/ambari/centos6/1.x/updates/1.7.0/ambari.repo -O /etc/yum.repos.d/ambari.repo本次安装使用这版本
wget http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.1.0/ambari.repo (这个是ambari2.1的，这个版本没有ganglia和nagios了，改为了ambari Metrics)

yum install ambari-server
(http://public-repo-1.hortonworks.com/ambari/centos6/1.x/updates/1.7.0/ambari/ambari-server-1.7.0-169.noarch.rpm)

三、安装MySQL服务
1）安装
通过yum安装mysql：
[root@ambari ~]# yum install mysql-server
启动mysql服务
[root@ambari ~]# service mysqld start
 
安装MySQL JDBC Connector：
[root@ambari root]# yum install mysql-connector-java

2）为root用户设置新密码等初始化工作
[root@ambari ~]# /usr/bin/mysql_secure_installation

3）为ambari创建数据库，配置相应用户和权限
[root@ambari ~]# mysql -u root -p
mysql> create database ambari;
Query OK, 1 row affected (0.00 sec)

4)hive库
mysql> create database hive;
mysql> create user "hive" identified by "hive123";
mysql> grant all privileges on hive.* to hive;
mysql> flush privileges;
 
5)启动
mysql设置为开机自启动
[root@cluster-05 root]$ chkconfig mysqld on
[root@cluster-05 root]$ chkconfig -list
mysqld             0:off      1:off      2:on      3:on      4:on      5:on      6:off

6)导入数据库
注意，此刻需要切换到mysql中执行相应脚本操作。
需要登录到mysql数据库，执行以下脚本Ambari-DDL-MySQL-CREATE.sql
[root@ambari ~]# mysql -u root -p
mysql> use ambari;
Database changed
mysql> source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
执行脚本，成功。

验证脚本是否初始化成功，出现以下table列表。
mysql> show tables;
+-------------------------------+
| Tables_in_ambari              |
+-------------------------------+
| ClusterHostMapping            |
| QRTZ_BLOB_TRIGGERS            |
| QRTZ_CALENDARS                |
| QRTZ_CRON_TRIGGERS            |
| QRTZ_FIRED_TRIGGERS           |
| QRTZ_JOB_DETAILS              |
| QRTZ_LOCKS                    |

ambari权限设置
CREATE USER 'ambari'@'%' IDENTIFIED BY 'bigdata';
GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'%';
FLUSH PRIVILEGES;

四、ambari server安装（此步骤会从官网下载包文件，如果失败就多重试几次，如果没有网络或网络很差，参考最后连接的有离线安装方式）
[root@Ambari yum.repos.d]# ambari-server setup
Using python  /usr/bin/python2.6
Setup ambari-server
Checking SELinux...
SELinux status is 'disabled'
Customize user account for ambari-server daemon [y/n] (n)? （直接回车）
Adjusting ambari-server permissions and ownership...
Checking firewall...
WARNING: iptables is running. Confirm the necessary Ambari ports are accessible. Refer to the Ambari documentation for more details on ports.
OK to continue [y/n] (y)? （直接回车）
Checking JDK...
[1] - Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7
[2] - Oracle JDK 1.6 + Java Cryptography Extension (JCE) Policy Files 6
[3] - Custom JDK
==============================================================================
Enter choice (1):1 （直选择1）
To download the Oracle JDK and the Java Cryptography Extension (JCE) Policy Files you must accept the license terms found at
http://www.oracle.com/technetwork/java/javase/terms/license/index.html and not accepting will cancel the Ambari Server setup and you must install the JDK and JCE files manually.
Do you accept the Oracle Binary Code License Agreement [y/n] (y)? （直接回车）
Downloading JDK from http://public-repo-1.hortonworks.com/ARTIFACTS/jdk-7u67-linux-x64.tar.gz to /var/lib/ambari-server/resources/jdk-7u67-linux-x64.tar.gz
JDK distribution size is 142376665 bytes
jdk-7u67-linux-x64.tar.gz... 100% (135.8 MB of 135.8 MB)
Successfully downloaded JDK distribution to /var/lib/ambari-server/resources/jdk-7u67-linux-x64.tar.gz
Installing JDK to /usr/jdk64
Successfully installed JDK to /usr/jdk64/jdk1.7.0_67
Downloading JCE Policy archive from http://public-repo-1.hortonworks.com/ARTIFACTS/UnlimitedJCEPolicyJDK7.zip to /var/lib/ambari-
server/resources/UnlimitedJCEPolicyJDK7.zip
Successfully downloaded JCE Policy archive to /var/lib/ambari-server/resources/UnlimitedJCEPolicyJDK7.zip
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)? y （y回车）
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL
[4] - PostgreSQL
==============================================================================
Enter choice (1): 3
Hostname (localhost): （直接回车）
Port (3306): （直接回车）
Database Name (ambari): （直接回车）
Username (ambari): （直接回车）
Enter Database Password (bigdata): （输入ambari密码，我这里是默认bigdata）
Copying JDBC drivers to server resources...
Configuring remote database connection properties...
WARNING: Before starting Ambari Server, you must run the following DDL against the database to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
Proceed with configuring remote database connection properties [y/n] (y)? （直接回车）
Copying JDBC drivers to server resources...
Extracting system views...
ambari-admin-1.7.0.169.jar
..
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.

[root@ambari ~]# ambari-server start
Using python  /usr/bin/python2.6
Starting ambari-server
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start....................
Ambari Server 'start' completed successfully.

使用浏览器打开IP:8080, 输入admin/admin 即可访问ambari管理界面开始HDP集群安装。

五、安装HDP（包含hadoop、zookeeper、hive等等）
此内容参考http://www.cnblogs.com/tgzhu/tag/HDP/ 写的挺详细了

问题一：
启动服务时报错：
Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py", line 37, in <module>
    BeforeStartHook().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 123, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py", line 27, in hook
    import params
  File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py", line 114, in <module>
    ambari_db_rca_password = config['hostLevelParams']['ambari_db_rca_password'][0]
TypeError: 'int' object is unsubscriptable


修改文件
vi /var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py
注释
#ambari_db_rca_password = config['hostLevelParams']['ambari_db_rca_password'][0]

貌似个是bug
https://community.hortonworks.com/questions/37772/file-varlibambari-agentcachestackshdp206hooksbefor.html

问题二：
在root下使用sudo启动，报错：

webhcat: starting ...
webhcat: /usr/local/hadoop/bin/hadoop jar /usr/local/apache-hive-0.14.0-bin/hcatalog/sbin/../share/webhcat/svr/lib/hive-webhcat-0.14.0.jar org.apache.hive.hcatalog.templeton.Main  
/usr/local/hive/hcatalog/sbin/webhcat_server.sh: line 143: ./webhcat-console.log: Permission denied
/usr/local/apache-hive-0.14.0-bin/hcatalog/sbin/webhcat_server.sh: failed to start. Check logs in  .

原因：hive默认在启动用户的home目录下生成日志文件，这时候需要修改hive的log4j的配置文件！

vim $HIVE_HOME/hcatalog/etc/webhcat/webhcat-log4j.properties
webhcat.log.dir = /var/hive/logs

参考：
http://www.julyme.com/20161104/15.html
http://blog.csdn.net/wind520/article/details/40193353
http://www.sjsjw.com/kf_cloud/article/022598ABA025291.asp