自建hadoop yum 仓库

Hadoop day 1 Apache Ambari 部署 里面安装组件经常失败，因为ambari 安装组件都是从yum仓库下载软件，但是国外的仓库由于某种不可抗拒的因素…你懂的。所以今天要建立一个自己的hadoop仓库。

仓库

方式1：

找台web服务器  ，新建一repo个文件

$ vim /etc/yum.repos.d/cloudera-cdh4.repo
[cloudera-cdh4]
name=cdh4
baseurl = http://archive.cloudera.com/cdh4/redhat/6/x86_64/cdh/4/
gpgkey = http://archive.cloudera.com/cdh4/redhat/6/x86_64/cdh/RPM-GPG-KEY-cloudera
gpgcheck = 1

存放rpm的物理目录 并同步

mkdir -p /opt/cdh/cloudera-cdh4 && cd /opt/cdh/cloudera-cdh4
reposync -r cloudera-cdh4
mkdir /data0/www/cdh4/el6

下载好的rpm 都迁走

mv /opt/cdh/cloudera-cdh4/RPMS/x86_64/* /data0/www/cdh4/el6  
mv /opt/cdh/cloudera-cdh4/RPMS/noarch/* /data0/www/cdh4/el6

创建元数据

cd /data0/www/cdh4/el6
createrepo .

做一个域名repo.t.nxin.com，location /cdh4/el6 指向  /data0/www/cdh4/

 

方式1，太慢，换方式2

将下面3个压缩包下载到/data0/www/HDP/， 版本选用ambari 界面选的 HDP2.3

http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.3.2.0/HDP-2.3.2.0-centos6-rpm.tar.gz
http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/centos6/HDP-UTILS-1.1.0.20-centos6.tar.gz
http://public-repo-1.hortonworks.com/ambari/centos6/ambari-1.5.1-centos6.tar.gz

解压

tar xf HDP-2.1-latest-centos6-rpm.tar.gz
tar xf HDP-UTILS-1.1.0.17-centos6.tar.gz
tar xf ambari-1.5.1-centos6.tar.gz

 

节点

每个节点新建一个仓库

$ vim /etc/yum.repos.d/cloudera-cdh4.repo
[cloudera-cdh4]
name=Cloudera’s Distribution for Hadoop, Version 4
baseurl= http://repo.t.nxin.com/cdh4/el6/     #web服务器地址 
gpgcheck =0

[Ambari-1.5.1]
name=Ambari-1.5.1
baseurl=http://repo.t.nxin.com/repo/ambari/centos6/1.x/updates/1.5.1
gpgcheck=0
enabled=1

 

vim /etc/yum.repos.d/HDP.repo
[HDP-2.3]
name=HDP-2.3
baseurl=http://repo.t.nxin.com/HDP/centos6/2.x/updates/2.3.2.0
path=/
enabled=1
gpgcheck=0

vim /etc/yum.repos.d/HDP-UTILS.repo
[HDP-UTILS-1.1.0.20]
name=HDP-UTILS-1.1.0.20
baseurl=http://repo.t.nxin.com/HDP-UTILS-1.1.0.20/repos/centos6
gpgcheck=0
enabled=1

vim /etc/yum.repos.d/ambari.repo
[Updates-ambari-2.1.2]
name=ambari-2.1.2 - Updates
baseurl=http://repo.t.nxin.com/ambari-2.1.2.1/centos6
gpgcheck=1
gpgkey=http://repo.t.nxin.com/ambari-2.1.2.1/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1

然后测试一下

yum clean all
yum search hadoop ambari

这样以后就可以愉快地 yum install xxx 了

 

如果手动安装的话执行下面的命令

yum install "hadoop_2.3.2_2950*" "oozie_2.3.2_2950*" "pig_2.3.2_2950*" "sqoop_2.3.2_2950*" "zookeeper