CDH5安装

https://wiki.debian.org/JavaPackage

http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/latest/CDH5-Installation-Guide/cdh5ig_cdh5_install.html?scroll=topic_4_4_2_unique_1

java-package provides the ability to build a Debian package from a Java binary distribution by running make-jpkg <java binary archive file> (with archive files downloaded from providers listed below). Should your interest be more on the development with Java under Debian, then follow the Java link. To learn more about the packaging of Java-written programs and libraries for Debian, see Teams/JavaPackaging.

The package includes the make-jpkg command to do this. Typical usage consists in:

• downloading one of the java binary archive listed below
• invoking make-jpkg to build a Debian package from the downloaded archive
• installing the generated package

Supported Java binary distributions currently include:

• Oracle (http://www.oracle.com/technetwork/java/javase/downloads):

  • The J2SE Development Kit (JDK), version 6 (update >= 10), 7

  • The J2SE Runtime Environment (JRE), version 6 (update >= 10), 7

  • The J2SE API Javadoc, version 6 (update >= 10), 7

---

NOTE: All other previously supported JVM (Blackdown, IBM, ..) have been removed and will only be added back on users demand.

---

Process

1. Add a "contrib" component to /etc/apt/sources.list, for example:

   # Debian 7 "Wheezy"
   deb http://http.debian.net/debian/ wheezy main contrib

2. Update the list of available packages and install the java-package package:

   # apt-get update && apt-get install java-package && exit

3. Download the desired Java JDK/JRE binary distribution (Oracle). Choose tar.gz archives or self-extracting archives, do not choose the RPM!

4. Use java-package to create a Debian package, for example:

   $ make-jpkg jdk-7u45-linux-x64.tar.gz

5. Install the binary package created:

   $ su
   # dpkg -i oracle-j2sdk1.7_1.7.0+update45_amd64.deb

Configuration

By default the DebianAlternatives will automatically install the best version of Java as the default version. If the symlinks have been manually set they will be preserved by the tools. The update-alternatives tools try hard to respect explicit configuration from the local admin. Local manual symlinks appear to be an explicit configuration. In order to reset the alternative symlinks to their default value use the --auto option.

# update-alternatives --auto java

If you'd like to override the default to perhaps use a specific version then use --config and manually select the desired version.

# update-alternatives --display java
# update-alternatives --config java 

Choose the appropriate number for the desired alternative.

The appropriate java binary will automatically be in PATH by virtue of the /usr/bin/java alternative symlink.

You may as well use the update-alternatives tool from java-common package which let you update all alternatives belonging to one runtime or development kit at a time.

# update-java-alternatives -l
# update-java-alternatives -s j2sdk1.7-oracle


Installing CDH 5
 
 
    
  
  
  
  
Add or Build the CDH 5 Repository or Download the      "1-click Install" package. 
Optionally Add a Repository Key
Install CDH 5 with YARN
Install CDH 5 with MRv1
(Optional) Install LZO
Deploy CDH and Install Components
  
 
 
 
 

  
  
Add or Build the CDH 5 Repository or Download the      "1-click Install" package. 
  
  
      
   
   
If you are installing CDH 5 on a Red Hat system, you can download Cloudera packagesusing yum or your web browser.
If you are installing CDH 5 on a SLES system, you can download the Cloudera packagesusing zypper or YaST or your web browser.
If you are installing CDH 5 on an Ubuntu or Debian system, you can download the Clouderapackages using apt or your web browser.
      
   
   

    
    
On Red Hat-compatible Systems
                
    
    

     
      Use one of the following methods to add or build the CDH 5 repository          or download the package on Red Hat-compatible systems. 
     
     

      
       Note:             
      
      
Use only one of the three methods.
          
     
     
    
    
        
    
    
Download and install the CDH 5 "1-click Install" package            OR          
Add the CDH 5 repository            OR          
Build a Yum Repository          
        
    
    
          Do this on all the systems in the cluster.        
        
    
    
          To download and install the CDH 5 "1-click Install" package:        
        
    
    
Click the entry in the table below that matches your Red Hat or            CentOS system, choose Save File, and save the file to            a directory to which you have write access (it can be your home directory). 
      
      
 
       
For OS Version
Click this Link
Red Hat/CentOS/Oracle 5
Red Hat/CentOS/Oracle 5 link
Red Hat/CentOS/Oracle 6 (64-bit)
Red Hat/CentOS/Oracle 6 link (64-bit)
 
      
          
Install the RPM. For Red Hat/CentOS/Oracle 5:              
$ sudo yum --nogpgcheck localinstall cloudera-cdh-5-0.x86_64.rpm  
 For Red Hat/CentOS/Oracle 6 (64-bit): 
$ sudo yum --nogpgcheck localinstall cloudera-cdh-5-0.x86_64.rpm
          
        
    
    
 Now continue with Step 1a: Optionally Add a Repository Key, and then choose            Install CDH 5 with YARN, or Install CDH 5 with MRv1;          or do both steps if you want to install both implementations. 
        
    
    
          OR: To add the CDH 5 repository:        
        
    
    
          Click the entry in the table below that matches your Red Hat or CentOS system, navigate to the repo file for          your system and save it in the /etc/yum.repos.d/ directory.        
        
    
    

     
     
For OS Version 
Click this Link 
Red Hat/CentOS/Oracle 5 
Red Hat/CentOS/Oracle 5 link 
Red Hat/CentOS/Oracle 6 (64-bit) 
Red Hat/CentOS/Oracle 6 link 
    
    
        
    
    
 Now continue with Step 1a: Optionally Add a Repository Key, and then choose            Install CDH 5 with YARN, or Install CDH 5 with MRv1;          or do both steps if you want to install both implementations. 
        
    
    
          OR: To build a Yum repository:        
        
    
    
          If you want to create your own yum repository, download the appropriate repo file, create the          repo, distribute the repo file and set up a web server, as described under          Creating a Local Yum Repository.        
        
    
    
 Now continue with Step 1a: Optionally Add a Repository Key, and then choose            Install CDH 5 with YARN, or Install CDH 5 with MRv1;          or do both steps if you want to install both implementations. 
      
   
   
      
   
   

    
    
On SLES Systems
                
    
    

     
      Use one of the following methods to download the CDH 5 repository or          package on SLES systems.
     
     

      
       Note:             
      
      
Use only one of the three methods.
          
     
     
    
    
        
    
    
Download and install the CDH 5 "1-click Install" PackageOR          
Add the CDH 5 repositoryOR          
Build a SLES Repository          
        
    
    
          To download and install the CDH 5 "1-click Install" package:        
        
    
    
Download the CDH 5 "1-click Install" package.
Click this link, choose Save File, and save it to a directory to which you have write access (it can be your home directory).
Install the RPM: 
$ sudo rpm -i cloudera-cdh-5-0.x86_64.rpm
Update your system package index by running: 
$ sudo zypper refresh
        
    
    
 Now continue with Step 1a: Optionally Add a Repository Key, and then choose            Install CDH 5 with YARN, or Install CDH 5 with MRv1;          or do both steps if you want to install both implementations. 
        
    
    
          OR: To add the CDH 5 repository:        
        
    
    
Run the following command:            
$ sudo zypper addrepo -f http://archive.cloudera.com/cdh5/sles/11/x86_64/cdh/cloudera-cdh5.repo
          
Update your system package index by running:
$ sudo zypper refresh
          
        
    
    
 Now continue with Step 1a: Optionally Add a Repository Key, and then choose            Install CDH 5 with YARN, or Install CDH 5 with MRv1;          or do both steps if you want to install both implementations. 
        
    
    
          OR: To build a SLES repository:        
        
    
    
          If you want to create your own SLES repository, create a mirror of          the CDH          SLES directory by following          these          instructions that explain how to create a SLES repository from the mirror.        
        
    
    
 Now continue with Step 1a: Optionally Add a Repository Key, and then choose            Install CDH 5 with YARN, or Install CDH 5 with MRv1;          or do both steps if you want to install both implementations. 
      
   
   
      
   
   

    
    
On Ubuntu or Debian Systems
                
    
    

     
      Use one of the following methods to download the CDH 5 repository or package.
     
     

      
       Note:             
      
      
Use only one of the three methods.
          
     
     
    
    
        
    
    
Download and install the CDH 5 "1-click Install" Package            OR          
Add the CDH 5 repositoryOR          
Build a DebianRepository          
        
    
    
           To download and install the CDH 5 "1-click Install" package:        
        
    
    
Download the CDH 5 "1-click Install" package:
Clickone of the following: this link for aWheezy system, or this link for aPrecise system.
Install the package. Do one of the following: Choose Open with in the download window to use the package manager, or Choose Save File, save the package to a directory to which you have write access (it can be your home directory) and install it from the command line, for example: 
sudo dpkg -i cdh5-repository_1.0_all.deb
        
    
    
 Now continue with Step 1a: Optionally Add a Repository Key, and then choose            Install CDH 5 with YARN, or Install CDH 5 with MRv1;          or do both steps if you want to install both implementations. 
        
    
    
          OR: To add the CDH 5 repository:        
        
    
    
          Create a new file /etc/apt/sources.list.d/cloudera.list with the following contents:        
        
    
    
For Ubuntu systems:            
deb [arch=amd64] http://archive.cloudera.com/cdh5/<OS-release-arch><RELEASE>-cdh5 contrib 
deb-src http://archive.cloudera.com/cdh5/<OS-release-arch><RELEASE>-cdh5 contrib
          
For Debian systems:            
deb http://archive.cloudera.com/cdh5/<OS-release-arch><RELEASE>-cdh5 contrib
deb-src http://archive.cloudera.com/cdh5/<OS-release-arch><RELEASE>-cdh5 contrib
          
        
    
    
 where: <OS-release-arch> is debian/wheezy/amd64/cdh or ubuntu/precise/amd64/cdh, and <RELEASE>          is the name of your distribution, which you can find by running lsb_release -c. 
        
    
    
 For example, to install CDH 5 for 64-bit Ubuntu Precise: 
        
    
    
deb [arch=amd64] http://archive.cloudera.com/cdh5/ubuntu/precise/amd64/cdh precise-cdh5 contrib 
deb-src http://archive.cloudera.com/cdh5/ubuntu/precise/amd64/cdh precise-cdh5 contrib
        
    
    
 Now continue with Step 1a: Optionally Add a Repository Key, and then choose            Install CDH 5 with YARN, or Install CDH 5 with MRv1;          or do both steps if you want to install both implementations. 
        
    
    
          OR: To build a Debian repository:        
        
    
    
          If you want to create your own apt repository, create a mirror of          the CDH Debian          directory and then          create          an apt repository from the mirror.        
        
    
    
 Now continue with Step 1a: Optionally Add a Repository Key, and then choose            Install CDH 5 with YARN, or Install CDH 5 with MRv1;          or do both steps if you want to install both implementations. 
      
   
   
    
  
  
 
 
 
 

  
  
Optionally Add a Repository Key
  
  
      
   
   
        Before installing YARN or MRv1: (Optionally) add a        repository key on each system in the cluster. Add the Cloudera Public GPG Key to your        repository by executing one of the following commands: 
      
   
   
For Red Hat/CentOS/Oracle 5            systems:          
$ sudo rpm --import http://archive.cloudera.com/cdh5/redhat/5/x86_64/cdh/RPM-GPG-KEY-cloudera
        
For Red Hat/CentOS/Oracle 6            systems:          
$ sudo rpm --import http://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/RPM-GPG-KEY-cloudera
        
For all SLES systems:          
$ sudo rpm --import http://archive.cloudera.com/cdh5/sles/11/x86_64/cdh/RPM-GPG-KEY-cloudera
        
For Ubuntu Lucid systems:          
$ curl -s http://archive.cloudera.com/cdh5/ubuntu/lucid/amd64/cdh/archive.key
| sudo apt-key add -
        
For Ubuntu Precise systems:          
$ curl -s http://archive.cloudera.com/cdh5/ubuntu/precise/amd64/cdh/archive.key
| sudo apt-key add -
        
For Debian Squeeze systems:          
$ curl -s http://archive.cloudera.com/cdh5/debian/squeeze/amd64/cdh/archive.key
| sudo apt-key add -
        
      
   
   
 This key enables you to verify that you are downloading genuine        packages. 
    
  
  
 
 
 
 

  
  
Install CDH 5 with YARN
  
  
      
   
   

    
     Note:         
    
    
 Skip this step if you intend to use only MRv1. Directions for installing MRv1 are in Step 3. 
      
   
   
      
   
   
        To install CDH 5 with YARN:      
      
   
   

    
     Note:         
    
    
If you decide to configure HA            for the NameNode, do not install hadoop-hdfs-secondarynamenode. After completing the HA            software configuration, follow the installation instructions under Deploying HDFS High Availability.
      
   
   
      
   
   
Install and deploy ZooKeeper.
     
     

      
       Important:             
      
      
 Cloudera recommends that you install (or update) and start a ZooKeeper cluster before proceeding. This is a requirement if you are deploying high availability (HA) for the NameNode or JobTracker. 
          
     
     
          
            Follow instructions under ZooKeeper Installation.          
        
Install each type of daemon package on the appropriate systems(s), asfollows.
     
     

      
      
Where to install 
Install commands 
Resource Manager host (analogous to MRv1 JobTracker) running: 
  
Red Hat/CentOS compatible 
sudo yum clean all; sudo yum install hadoop-yarn-resourcemanager 
SLES 
sudo zypper clean --all; sudo zypper install hadoop-yarn-resourcemanager 
Ubuntu or Debian 
sudo apt-get update; sudo apt-get install hadoop-yarn-resourcemanager 
NameNode host running: 
  
Red Hat/CentOS compatible 
sudo yum clean all; sudo yum install hadoop-hdfs-namenode 
SLES 
sudo zypper clean --all; sudo zypper install hadoop-hdfs-namenode 
Ubuntu or Debian 
sudo apt-get install hadoop-hdfs-namenode 
Secondary NameNode host (if used) running: 
  
Red Hat/CentOS compatible 
sudo yum clean all; sudo yum install hadoop-hdfs-secondarynamenode 
SLES 
sudo zypper clean --all; sudo zypper install hadoop-hdfs-secondarynamenode 
Ubuntu or Debian 
sudo apt-get install hadoop-hdfs-secondarynamenode 
All cluster hosts except the Resource Manager running: 
  
Red Hat/CentOS compatible 
sudo yum clean all; sudo yum install hadoop-yarn-nodemanager hadoop-hdfs-datanode hadoop-mapreduce 
SLES 
sudo zypper clean --all; sudo zypper install hadoop-yarn-nodemanager hadoop-hdfs-datanode hadoop-mapreduce 
Ubuntu or Debian 
sudo apt-get install hadoop-yarn-nodemanager hadoop-hdfs-datanode hadoop-mapreduce 
One host in the cluster running: 
  
Red Hat/CentOS compatible 
sudo yum clean all; sudo yum install hadoop-mapreduce-historyserver hadoop-yarn-proxyserver 
SLES 
sudo zypper clean --all; sudo zypper install hadoop-mapreduce-historyserver hadoop-yarn-proxyserver 
Ubuntu or Debian 
sudo apt-get install hadoop-mapreduce-historyserver hadoop-yarn-proxyserver 
All client hosts running: 
  
Red Hat/CentOS compatible 
sudo yum clean all; sudo yum install hadoop-client 
SLES 
sudo zypper clean --all; sudo zypper install hadoop-client 
Ubuntu or Debian 
sudo apt-get install hadoop-client 
     
     
        
      
   
   

    
     Note:         
    
    
          The hadoop-yarn and hadoop-hdfs packages are installed on each system          automatically as dependencies of the other packages.        
      
   
   
    
  
  
 
 
 
 

  
  
Install CDH 5 with MRv1
  
  
      
   
   

    
     Note:         
    
    
If you are also installing YARN, you can skip any packages you have          already installed in Install CDH 5 with YARN.
        
    
    
 Skip this step and go to Install CDH 5 with YARN if you intend to use only          YARN. 
      
   
   
      
   
   

    
     Important:         
    
    
 Before proceeding, you need to decide: 
        
    
    
Whether to configure High Availability (HA) for the NameNode            and/or JobTracker; see the  CDH 5 High Availability Guide for more            information and instructions.
Where to deploy the NameNode, Secondary NameNode, and JobTracker            daemons. As a general rule:
      
      
The NameNode and JobTracker run on the the same "master" host unless the cluster is large (more than a few tens of nodes), and the master host (or hosts) should not run the Secondary NameNode (if used), DataNode or TaskTracker services.
In a large cluster, it is especially important that the Secondary NameNode (if used) runs on a separate machine from the NameNode.
Each node in the cluster except the master host(s) should run the DataNode and TaskTracker services.
          
        
    
    
If you decide to configure HA for the            NameNode, do not install hadoop-hdfs-secondarynamenode. After completing the HA            software configuration, follow the installation instructions under Deploying HDFS High Availability.
      
   
   
      
   
   
Install and deploy ZooKeeper. 
     
     

      
       Important:             
      
      
 Cloudera recommends that you install (or update) and start a ZooKeeper cluster before proceeding. This is a requirement if you are deploying high availability (HA) for the NameNode or JobTracker. 
          
     
     
          
 Follow instructions under ZooKeeper Installation. 
        
Install each type of daemon package on the appropriate systems(s),          as follows. 
     
     

      
      
Where to install 
Install commands 
JobTracker host running: 
  
Red Hat/CentOS compatible 
sudo yum clean all; sudo yum install hadoop-0.20-mapreduce-jobtracker 
SLES 
sudo zypper clean --all; sudo zypper install hadoop-0.20-mapreduce-jobtracker 
Ubuntu or Debian 
sudo apt-get update; sudo apt-get install hadoop-0.20-mapreduce-jobtracker 
NameNode host running: 
  
Red Hat/CentOS compatible 
sudo yum clean all; sudo yum install hadoop-hdfs-namenode 
SLES 
sudo zypper clean --all; sudo zypper install hadoop-hdfs-namenode 
Ubuntu or Debian 
sudo apt-get install hadoop-hdfs-namenode 
Secondary NameNode host (if used) running: 
  
Red Hat/CentOS compatible 
sudo yum clean all; sudo yum install hadoop-hdfs-secondarynamenode 
SLES 
sudo zypper clean --all; sudo zypper install hadoop-hdfs-secondarynamenode 
Ubuntu or Debian 
sudo apt-get install hadoop-hdfs-secondarynamenode 
All cluster hosts except the JobTracker, NameNode, and Secondary (or Standby) NameNode hosts running: 
  
Red Hat/CentOS compatible 
sudo yum clean all; sudo yum install hadoop-0.20-mapreduce-tasktracker hadoop-hdfs-datanode 
SLES 
sudo zypper clean --all; sudo zypper install hadoop-0.20-mapreduce-tasktracker hadoop-hdfs-datanode 
Ubuntu or Debian 
sudo apt-get install hadoop-0.20-mapreduce-tasktracker hadoop-hdfs-datanode 
All client hosts running: 
  
Red Hat/CentOS compatible 
sudo yum clean all; sudo yum install hadoop-client 
SLES 
sudo zypper clean --all; sudo zypper install hadoop-client 
Ubuntu or Debian 
sudo apt-get install hadoop-client 
     
     
        
    
  
  
 
 
 
 

  
  
(Optional) Install LZO
  
  
      
   
   
If you decide to install LZO ( Lempel–Ziv–Oberhumer compression), proceed as follows.
      
   
   
Add the repository on each node in the cluster.
     
     

      
      Follow the instructions for your OS              version:
      
      
 
       
For OS Version 
Do this
Red Hat/CentOS/Oracle 5 
Navigate to this link and save the file in the /etc/yum.repos.d/ directory. 
Red Hat/CentOS 6 
Navigate to this link and save the file in the /etc/yum.repos.d/ directory.
SLES
 
           
Run the following command:
 $ sudo zypper addrepo -f 
http://archive.cloudera.com/gplextras5/sles/11/x86_64/gplextras/
cloudera-gplextras5.repo
Update your system package index by running:
 $ sudo zypper refresh
Ubuntu or Debian
Navigate to this link and save the file as /etc/apt/sources.list.d/gplextras.list. 
           
 
             Important: Make sure you do not let the file name default to 
            cloudera.list, as that will overwrite your existing 
            cloudera.list. 
           
 
      
     
     
Install the package on each node as follows: 
     
     

      
      
For OS version 
Install commands 
Red Hat/CentOS compatible 
 
          
 
           
sudo yum install hadoop-lzo
 
          
SLES 
 
          
 
           
sudo zypper install hadoop-lzo
 
          
Ubuntu or Debian 
 
          
 
           
sudo apt-get install hadoop-lzo
 
          
     
     
 Continue with installing and deploying CDH. As part of the          deployment, you will need to do some additional configuration for LZO, as shown under             Configuring LZO . 
     
     

      
       Important: Make sure you do            this configuration 
      
      after you have 
      
      copied the default configuration files to a custom            location and set alternatives to point to it.
     
     
    
  
  
 
 
 
 

  
  
Deploy CDH and Install Components
  
  
      
   
   
        Now proceed with:      
      
   
   
deploying CDH 5        
installing components.
    
  
  
 
 

---

CategoryJava