Hadoop 管理监控工具 Apache Ambari
Apache Ambari是一种基于Web的工具,支持Apache Hadoop集群的供应、管理和监控。Ambari目前已支持大多数Hadoop组件,包括HDFS、MapReduce、Hive、Pig、 Hbase、Zookeper、Sqoop和Hcatalog等。
Apache Ambari 支持HDFS、MapReduce、Hive、Pig、Hbase、Zookeper、Sqoop和Hcatalog等的集中管理。也是5个顶级hadoop管理工具之一。
Step1: Download the Ambari repository on the Ambari Server host
[root@jifeng02 ~]# cd /etc/yum.repos.d/
Choose an appropriate <ambari-repo-url> based on the platform used:
[root@jifeng02 yum.repos.d]# wget http://public-repo-1.hortonworks.com/ambari/centos6/1.x/updates/1.6.1/ambari.repo
--2014-10-17 22:30:35-- http://public-repo-1.hortonworks.com/ambari/centos6/1.x/updates/1.6.1/ambari.repo
正在解析主机 public-repo-1.hortonworks.com... 54.192.157.143, 54.230.158.152, 54.230.158.197, ...
正在连接 public-repo-1.hortonworks.com|54.192.157.143|:80... 已连接。
已发出 HTTP 请求,正在等待回应... 200 OK
长度:472 [binary/octet-stream]
正在保存至: “ambari.repo”
100%[======================================================================>] 472 --.-K/s in 0s
2014-10-17 22:30:35 (81.0 MB/s) - 已保存 “ambari.repo” [472/472])
Step 2: Install, Setup, and Start Ambari Server
yum install ambari-server[root@jifeng02 jifeng]# yum install ambari-server
Loaded plugins: fastestmirror, refresh-packagekit, security
Loading mirror speeds from cached hostfile
* base: centos.ustc.edu.cn
* extras: centos.ustc.edu.cn
* updates: centos.ustc.edu.cn
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package ambari-server.noarch 0:1.6.1-98 will be installed
--> Processing Dependency: postgresql-server >= 8.1 for package: ambari-server-1.6.1-98.noarch
--> Running transaction check
---> Package postgresql-server.i686 0:8.4.20-1.el6_5 will be installed
--> Processing Dependency: postgresql-libs(x86-32) = 8.4.20-1.el6_5 for package: postgresql-server-8.4.20-1.el6_5.i686
--> Processing Dependency: postgresql(x86-32) = 8.4.20-1.el6_5 for package: postgresql-server-8.4.20-1.el6_5.i686
--> Processing Dependency: libssl.so.10(libssl.so.10) for package: postgresql-server-8.4.20-1.el6_5.i686
--> Processing Dependency: libcrypto.so.10(libcrypto.so.10) for package: postgresql-server-8.4.20-1.el6_5.i686
--> Running transaction check
---> Package openssl.i686 0:1.0.0-27.el6 will be updated
--> Processing Dependency: openssl = 1.0.0-27.el6 for package: openssl-devel-1.0.0-27.el6.i686
---> Package openssl.i686 0:1.0.1e-30.el6_5.2 will be an update
---> Package postgresql.i686 0:8.4.13-1.el6_3 will be updated
--> Processing Dependency: postgresql(x86-32) = 8.4.13-1.el6_3 for package: postgresql-devel-8.4.13-1.el6_3.i686
---> Package postgresql.i686 0:8.4.20-1.el6_5 will be an update
---> Package postgresql-libs.i686 0:8.4.13-1.el6_3 will be updated
---> Package postgresql-libs.i686 0:8.4.20-1.el6_5 will be an update
--> Running transaction check
---> Package openssl-devel.i686 0:1.0.0-27.el6 will be updated
---> Package openssl-devel.i686 0:1.0.1e-30.el6_5.2 will be an update
---> Package postgresql-devel.i686 0:8.4.13-1.el6_3 will be updated
---> Package postgresql-devel.i686 0:8.4.20-1.el6_5 will be an update
--> Finished Dependency Resolution
Dependencies Resolved
================================================================================================================
Package Arch Version Repository Size
================================================================================================================
Installing:
ambari-server noarch 1.6.1-98 Updates-ambari-1.6.1 39 M
Installing for dependencies:
postgresql-server i686 8.4.20-1.el6_5 updates 3.4 M
Updating for dependencies:
openssl i686 1.0.1e-30.el6_5.2 updates 1.5 M
openssl-devel i686 1.0.1e-30.el6_5.2 updates 1.2 M
postgresql i686 8.4.20-1.el6_5 updates 2.6 M
postgresql-devel i686 8.4.20-1.el6_5 updates 810 k
postgresql-libs i686 8.4.20-1.el6_5 updates 205 k
Transaction Summary
================================================================================================================
Install 2 Package(s)
Upgrade 5 Package(s)
Total download size: 49 M
Is this ok [y/N]: y
Downloading Packages:
(1/7): ambari-server-1.6.1-98.noarch.rpm | 39 MB 09:05
(2/7): openssl-1.0.1e-30.el6_5.2.i686.rpm | 1.5 MB 00:01
(3/7): openssl-devel-1.0.1e-30.el6_5.2.i686.rpm | 1.2 MB 00:01
(4/7): postgresql-8.4.20-1.el6_5.i686.rpm | 2.6 MB 00:03
(5/7): postgresql-devel-8.4.20-1.el6_5.i686.rpm | 810 kB 00:01
(6/7): postgresql-libs-8.4.20-1.el6_5.i686.rpm | 205 kB 00:00
(7/7): postgresql-server-8.4.20-1.el6_5.i686.rpm | 3.4 MB 00:03
----------------------------------------------------------------------------------------------------------------
Total 89 kB/s | 49 MB 09:20
warning: rpmts_HdrFromFdno: Header V4 RSA/SHA1 Signature, key ID 07513cad: NOKEY
Retrieving key from http://public-repo-1.hortonworks.com/ambari/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
Importing GPG key 0x07513CAD:
Userid: "Jenkins (HDP Builds) <jenkin@hortonworks.com>"
From : http://public-repo-1.hortonworks.com/ambari/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
Is this ok [y/N]: y
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Updating : openssl-1.0.1e-30.el6_5.2.i686 1/12
Updating : postgresql-libs-8.4.20-1.el6_5.i686 2/12
Updating : openssl-devel-1.0.1e-30.el6_5.2.i686 3/12
Updating : postgresql-8.4.20-1.el6_5.i686 4/12
Installing : postgresql-server-8.4.20-1.el6_5.i686 5/12
Installing : ambari-server-1.6.1-98.noarch 6/12
Updating : postgresql-devel-8.4.20-1.el6_5.i686 7/12
Cleanup : postgresql-devel-8.4.13-1.el6_3.i686 8/12
Cleanup : postgresql-8.4.13-1.el6_3.i686 9/12
Cleanup : postgresql-libs-8.4.13-1.el6_3.i686 10/12
Cleanup : openssl-devel-1.0.0-27.el6.i686 11/12
Cleanup : openssl-1.0.0-27.el6.i686 12/12
Verifying : postgresql-8.4.20-1.el6_5.i686 1/12
Verifying : postgresql-devel-8.4.20-1.el6_5.i686 2/12
Verifying : openssl-1.0.1e-30.el6_5.2.i686 3/12
Verifying : ambari-server-1.6.1-98.noarch 4/12
Verifying : postgresql-server-8.4.20-1.el6_5.i686 5/12
Verifying : openssl-devel-1.0.1e-30.el6_5.2.i686 6/12
Verifying : postgresql-libs-8.4.20-1.el6_5.i686 7/12
Verifying : postgresql-8.4.13-1.el6_3.i686 8/12
Verifying : openssl-1.0.0-27.el6.i686 9/12
Verifying : postgresql-libs-8.4.13-1.el6_3.i686 10/12
Verifying : postgresql-devel-8.4.13-1.el6_3.i686 11/12
Verifying : openssl-devel-1.0.0-27.el6.i686 12/12
Installed:
ambari-server.noarch 0:1.6.1-98
Dependency Installed:
postgresql-server.i686 0:8.4.20-1.el6_5
Dependency Updated:
openssl.i686 0:1.0.1e-30.el6_5.2 openssl-devel.i686 0:1.0.1e-30.el6_5.2
postgresql.i686 0:8.4.20-1.el6_5 postgresql-devel.i686 0:8.4.20-1.el6_5
postgresql-libs.i686 0:8.4.20-1.el6_5
Complete!
Run the setup command to configure your Ambari Server, Database, JDK, LDAP, and other options:
[root@jifeng02 jifeng]# ambari-server setup
Using python /usr/bin/python2.6
Setup ambari-server
Checking SELinux...
SELinux status is 'enabled'
SELinux mode is 'enforcing'
Temporarily disabling SELinux
WARNING: SELinux is set to 'permissive' mode and temporarily disabled.
OK to continue [y/n] (y)? y
Customize user account for ambari-server daemon [y/n] (n)? y
Enter user account for ambari-server daemon (root):jifeng
Adjusting ambari-server permissions and ownership...
Checking iptables...
Checking JDK...
[1] - Oracle JDK 1.7
[2] - Oracle JDK 1.6
[3] - Custom JDK
==============================================================================
Enter choice (1): 1
To download the Oracle JDK you must accept the license terms found at http://www.oracle.com/technetwork/java/javase/terms/license/index.html and not accepting will cancel the Ambari Server setup.
Do you accept the Oracle Binary Code License Agreement [y/n] (y)? y
Downloading JDK from http://public-repo-1.hortonworks.com/ARTIFACTS/jdk-7u45-linux-x64.tar.gz to /var/lib/ambari-server/resources/jdk-7u45-linux-x64.tar.gz
JDK distribution size is 138094686 bytes
jdk-7u45-linux-x64.tar.gz... 2% (3.0 MB of 131.7 MB)^C
Aborting ... Keyboard Interrupt.
选择:1 会自动下载jdk,终止掉
再次继续
[root@jifeng02 ~]# ambari-server setup
Using python /usr/bin/python2.6
Setup ambari-server
Checking SELinux...
SELinux status is 'enabled'
SELinux mode is 'permissive'
WARNING: SELinux is set to 'permissive' mode and temporarily disabled.
OK to continue [y/n] (y)? y
Ambari-server daemon is configured to run under user 'jifeng'. Change this setting [y/n] (n)? y
Enter user account for ambari-server daemon (root):jifeng
Adjusting ambari-server permissions and ownership...
Checking iptables...
Checking JDK...
[1] - Oracle JDK 1.7
[2] - Oracle JDK 1.6
[3] - Custom JDK
==============================================================================
Enter choice (1): 3
WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME: /home/jifeng/jdk1.7.0_45
Validating JDK on Ambari Server...done.
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)? y
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL
[4] - PostgreSQL
==============================================================================
Enter choice (1): 3
Hostname (localhost): 10.6.3.241
Port (3306):
Database Name (ambari): ambari
Username (ambari): dss
Enter Database Password (bigdata):
Re-enter password:
WARNING: Before starting Ambari Server, you must copy the MySQL JDBC driver JAR file to /usr/share/java.
Press <enter> to continue.
ERROR: Before starting Ambari Server, you must copy the MySQL JDBC driver JAR file to /usr/share/java.
ERROR: Exiting with exit code -1.
REASON: Before starting Ambari Server, you must copy the MySQL JDBC driver JAR file to /usr/share/java.
需要copy mysql的jdbc包到/usr/share/java
再来:
[root@jifeng02 ~]# ambari-server setup
Using python /usr/bin/python2.6
Setup ambari-server
Checking SELinux...
SELinux status is 'enabled'
SELinux mode is 'permissive'
WARNING: SELinux is set to 'permissive' mode and temporarily disabled.
OK to continue [y/n] (y)? y
Ambari-server daemon is configured to run under user 'jifeng'. Change this setting [y/n] (n)? y
Enter user account for ambari-server daemon (root):jifeng
Adjusting ambari-server permissions and ownership...
Checking iptables...
Checking JDK...
Do you want to change Oracle JDK [y/n] (n)? y
[1] - Oracle JDK 1.7
[2] - Oracle JDK 1.6
[3] - Custom JDK
==============================================================================
Enter choice (1): 3
WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME: /home/jifeng/jdk1.7.0_45
Validating JDK on Ambari Server...done.
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)? y
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL
[4] - PostgreSQL
==============================================================================
Enter choice (1): 3
Hostname (localhost): 10.6.3.241
Port (3306):
Database Name (ambari): ambari
Username (ambari): dss
Enter Database Password (bigdata):
Re-enter password:
Copying JDBC drivers to server resources...
Configuring remote database connection properties...
WARNING: Before starting Ambari Server, you must run the following DDL against the database to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
Proceed with configuring remote database connection properties [y/n] (y)? y
Copying JDBC drivers to server resources...
Ambari Server 'setup' completed successfully.
成功了!!
Start Ambari Server:
否则出现如下错误:
[root@jifeng02 ~]# ambari-server start
Using python /usr/bin/python2.6
Starting ambari-server
Ambari Server running with 'root' privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Waiting for server start...
sh: line 0: ulimit: open files: cannot modify limit: 不允许的操作
ERROR: Exiting with exit code -1.
REASON: Ambari Server java process died with exitcode 255. Check /var/log/ambari-server/ambari-server.out for more information.
执行/var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql 后
启动:
[root@jifeng02 ~]# ambari-server start
Using python /usr/bin/python2.6
Starting ambari-server
Ambari Server running with 'root' privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Waiting for server start...
sh: line 0: ulimit: open files: cannot modify limit: 不允许的操作
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Ambari Server 'start' completed successfully.
Step 3: Deploy Cluster using Ambari Web UI
访问地址 http://jifeng02:8080.
用户名和密码都是: admin
参考:https://cwiki.apache.org/confluence/display/AMBARI/Install+Ambari+1.6.1+from+Public+Repositories