Hadoop回收站trash

转自：http://www.cnblogs.com/ggjucheng/archive/2012/04/18/2454683.html

Hadoop回收站trash，默认是关闭的。
习惯了window的同学，建议最好还是把它提前开开，否则误操作的时候，就欲哭无泪了

1.修改conf/core-site.xml,增加

<property> 
<name>fs.trash.interval</name> 
<value>1440</value> 
<description>Number of minutes between trash checkpoints. 
If zero, the trash feature is disabled. 
</description> 
</property>

 

默认是0.单位分钟。这里我设置的是1天（60*24）
删除数据rm后，会将数据move到当前文件夹下的.Trash目录

2.测试
1）新建目录input

hadoop/bin/hadoop fs -mkdir input

 

2）上传文件

root@master:/data/soft# hadoop/bin/hadoop fs -copyFromLocal /data/soft/file0* input

3）删除目录input

[root@master data]# hadoop fs -rmr input 
Moved to trash: hdfs://master:9000/user/root/input

4）参看当前目录

[root@master data]# hadoop fs -ls 
Found 2 items 
drwxr-xr-x - root supergroup 0 2011-02-12 22:17 /user/root/.Trash

 

发现input删除，多了一个目录.Trash

5）恢复刚刚删除的目录

[root@master data]# hadoop fs -mv /user/root/.Trash/Current/user/root/input /user/root/input

6）检查恢复的数据

[root@master data]# hadoop fs -ls input 
Found 2 items 
-rw-r--r-- 3 root supergroup 22 2011-02-12 17:40 /user/root/input/file01 
-rw-r--r-- 3 root supergroup 28 2011-02-12 17:40 /user/root/input/file02

 
6）删除.Trash目录（清理垃圾）

[root@master data]# hadoop fs -rmr .Trash 
Deleted hdfs://master:9000/user/root/.Trash

回收站简介
在HDFS里，删除文件时，不会真正的删除，其实是放入回收站/trash
回收站里的文件可以快速恢复。
可以设置一个时间阈值，当回收站里文件的存放时间超过这个阈值或是回收站被清空时，文件才会被彻底删除，并且释放占用的数据块。

Hadoop 的回收站trash功能默认是关闭的，所以需要在core-site.xml中手动开启

--手动开启trash功能，添加fs.trash.interval属性
[grid@rac2 hadoop-0.20.2]$ bin/hadoop dfs -ls /user/grid/.Trash
ls: Cannot access /user/grid/.Trash: No such file or directory.

[grid@rac2 hadoop-0.20.2]$ vi conf/core-site.xml 
<property> 
<name>fs.trash.interval</name>    
<value>1440</value>    
<description>Number of minutes between trash checkpoints.    
    If zero, the trash feature is disabled.
</description>    
</property>

注：fs.trash.interval 的含义是文件删除后保留时长，默认为0，单位为分钟，这里设的是1天（60*24）

--测试删除数据
[grid@rac2 hadoop-0.20.2]$ bin/hadoop dfs -ls in
Found 2 items
-rw-r--r--  2 grid supergroup        12 2012-11-26 15:31 /user/grid/in/test1.txt
-rw-r--r--  2 grid supergroup        13 2012-11-26 15:31 /user/grid/in/test2.txt
[grid@rac2 hadoop-0.20.2]$ bin/hadoop dfs -rmr in
Moved to trash: hdfs://gc:9000/user/grid/in
[grid@rac2 hadoop-0.20.2]$ bin/hadoop dfs -rm abc
Moved to trash: hdfs://gc:9000/user/grid/abc

--检查回收站数据，发现多出了.Trash目录
[grid@rac2 hadoop-0.20.2]$ bin/hadoop dfs -ls /user/grid/.Trash
Found 1 items
drwxr-xr-x  - grid supergroup          0 2012-11-28 15:49 /user/grid/.Trash/Current
[grid@rac2 hadoop-0.20.2]$ bin/hadoop dfs -ls /user/grid/.Trash/Current
Found 1 items
drwxr-xr-x  - grid supergroup          0 2012-11-28 15:49 /user/grid/.Trash/Current/user
[grid@rac2 hadoop-0.20.2]$ bin/hadoop dfs -ls /user/grid/.Trash/Current/user
Found 1 items
drwx------  - grid supergroup          0 2012-11-28 15:50 /user/grid/.Trash/Current/user/grid
[grid@rac2 hadoop-0.20.2]$ bin/hadoop dfs -ls /user/grid/.Trash/Current/user/grid
Found 2 items
-rw-r--r--  2 grid supergroup          0 2012-11-27 14:04 /user/grid/.Trash/Current/user/grid/abc
drwxr-xr-x  - grid supergroup          0 2012-11-26 15:31 /user/grid/.Trash/Current/user/grid/in
注：在HDFS上的回收站数据在 /user/$USER/.Trash/Current/user/$USER/目录下

--恢复回收站数据
[grid@rac2 hadoop-0.20.2]$ bin/hadoop dfs -mv /user/grid/.Trash/Current/user/grid/in /user/grid/in
[grid@rac2 hadoop-0.20.2]$ bin/hadoop dfs -ls in
Found 2 items
-rw-r--r--  2 grid supergroup        12 2012-11-26 15:31 /user/grid/in/test1.txt
-rw-r--r--  2 grid supergroup        13 2012-11-26 15:31 /user/grid/in/test2.txt

--清空回收站
清空回收站命令：hdfs dfs -expunge
[grid@rac2 hadoop-0.20.2]$ bin/hadoop dfs -ls .Trash/Current/user/grid
Found 1 items
-rw-r--r--  2 grid supergroup          0 2012-11-27 14:04 /user/grid/.Trash/Current/user/grid/abc
[grid@rac2 hadoop-0.20.2]$ bin/hadoop dfs -expunge
12/11/28 16:09:12 INFO fs.Trash: Created trash checkpoint: /user/grid/.Trash/1211281609
[grid@rac2 hadoop-0.20.2]$ bin/hadoop dfs -ls .Trash/Current/user/grid
ls: Cannot access .Trash/Current/user/grid: No such file or directory.