Hadoop安装LZO

目录

---

环境

安装&编译

配置Hadoop支持LZO

安装LZO命令

LZO命令说明

---

环境

• Centos 6.5 64位
• JDK1.8
• Hadoop-2.6.0-cdh5.7.0
• LZO2.1.0包
• hadoop-lzo
• Maven3.6.0

---

安装&编译

1.安装依赖并下载lzo包

[hadoop@192 sbin]$ sudo yum -y install lzo-devel zlib-devel gcc autoconf automake libtool
[hadoop@192 software]$ wget http://www.oberhumer.com/opensource/lzo/download/lzo-2.10.tar.gz

2.解压并编译LZO包

[hadoop@192 software]$ tar -zxvf lzo-2.10.tar.gz -C ~/source
[hadoop@192 source]$ cd lzo-2.10/
[hadoop@192 lzo-2.10]$ ./configure --enable-shared --prefix /usr/local/lzo-2.10
[hadoop@192 lzo-2.10]$ make && sudo make install

3.下载并解压Hadoop-lzo

[hadoop@192 software]$ wget https://github.com/twitter/hadoop-lzo/archive/master.zip
# 解压
[hadoop@192 software]$ unzip master -d ~/source

4.修改master下的pom添加cloudera仓库

[hadoop@192 hadoop-lzo-master]$ pwd
/home/hadoop/source/hadoop-lzo-master
[hadoop@192 hadoop-lzo-master]$ ll
total 76
-rw-rw-r--. 1 hadoop hadoop 35147 Oct 13  2017 COPYING
-rw-rw-r--. 1 hadoop hadoop 19753 Oct 13  2017 pom.xml
-rw-rw-r--. 1 hadoop hadoop 10170 Oct 13  2017 README.md
drwxrwxr-x. 2 hadoop hadoop  4096 Oct 13  2017 scripts
drwxrwxr-x. 4 hadoop hadoop  4096 Oct 13  2017 src
[hadoop@192 hadoop-lzo-master]$ vi pom.xml

<repository>
   <id>cloudera-repo</id>
   <url>https://repository.cloudera.com/artifactory/cloudera-repos</url>
</repository>

5.修改hadoop版本

<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <ha