Using the LZO Parcel

This section describes how to install and use the LZO parcel.
The Repository


Add the appropriate repository to Cloudera Manager’s list of parcel repositories. The HADOOP_LZO parcel will then become available on the parcel management screen. If required, the repository can be mirrored in the same way as the CDH repo. Public customer repository: http://archive.cloudera.com/gplextras/parcels/latest.
Activation


The HADOOP_LZO parcel can be downloaded/distributed/activated in the same way as the CDH parcel. Once activated, it will be necessary to reconfigure and restart services that intend to use lzo functionality.
MapReduce


The HADOOP_LZO parcel can be downloaded/distributed/activated in the same way as the CDH parcel. Once activated, it will be necessary to reconfigure and restart services that intend to use lzo functionality.


    Add the following entries to the MapReduce environment safety valve:
        HADOOP_CLASSPATH=/opt/cloudera/parcels/HADOOP_LZO/lib/hadoop/lib/*
        JAVA_LIBRARY_PATH=/opt/cloudera/parcels/HADOOP_LZO/lib/hadoop/lib/native
    Add the following entries to the MapReduce Client environment safety valve:
        HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/opt/cloudera/parcels/HADOOP_LZO/lib/hadoop/lib/*
        JAVA_LIBRARY_PATH=$JAVA_LIBRARY_PATH:/opt/cloudera/parcels/HADOOP_LZO/lib/hadoop/lib/native
    Restart MapReduce
    Redeploy MapReduce Client Configuration


Oozie


    Go to /var/lib/oozie on each Oozie server and symlink the hadoop lzo jar.
        /opt/cloudera/parcels/HADOOP_LZO/lib/hadoop/lib/hadooplzocdh40.4.15gplextras. jar
    Restart Oozie


HBase


    Add the following entries to the HBase environment safety valve:
        HBASE_CLASSPATH=/opt/cloudera/parcels/HADOOP_LZO/lib/hadoop/lib/*
        JAVA_LIBRARY_PATH=/opt/cloudera/parcels/HADOOP_LZO/lib/hadoop/lib/native
    Restart HBase


Impala (1.0 or later)


This only works with Impala 1.0 or later.


    Add the following entry to the Impala environment safety valve:
        LD_LIBRARY_PATH=/opt/cloudera/parcels/HADOOP_LZO/lib/impala/lib
    Restart Impala


Notes


    Any service that does not require the use of LZO need not be configured. For example, if you are not using HBase, you do not need to do anything to the safety valve.
    The Oozie step is required, with or without parcels. The only difference is where you find the LZO jar to copy/replace. The LZO jar may already be present in /var/lib/oozie. Replacing any existing jar with the parcel jar (as described abvoe) is strongly recommended.