集群测试
上传小文件到集群,随便选择一个小文件上传到hdfs的根目录
[shaozhiqi@hadoop102 hadoop-3.1.2]$ bin/hdfs dfs -put wcinput/wc.input /
[shaozhiqi@hadoop102 hadoop-3.1.2]$
验证是否上传成功,验证ok,Replication的3代表三个副本数
![image.png](https://i-blog.csdnimg.cn/blog_migrate/7f8da58761bc7e864472539a8ff80781.webp?x-image-process=image/format,png)
image.png
上传一个大文件如我们的hadoop安装包
[shaozhiqi@hadoop102 hadoop-3.1.2]$ bin/hdfs dfs -put /opt/software/hadoop-3.1.2.tar.gz /
![image.png](https://i-blog.csdnimg.cn/blog_migrate/93c9b6e7ed3c449d68682f2c03ff6702.webp?x-image-process=image/format,png)
image.png
查看每个文件在hdfs的副本情况和存储块block,默认每个block大小为128M
关于块block的说明:当hdfs上收到的文件大于128M,则该文件会被切分存储为多个块,多个块以块为单位存储在不用的datanode上。但是值的注意的是如果某个文件大小没有达到128M,则按文件大小存储,不会占用整个128M的空间。
点击:Wc.Input
![image.png](https://i-blog.csdnimg.cn/blog_migrate/7cde3f267fc0badd76954d6b0118be05.webp?x-image-process=image/format,png)
image.png
点击Hadoop-3.1.2.tar.gz
![image.png](https://i-blog.csdnimg.cn/blog_migrate/8e6f3396b26e223500785fd839bf50fc.webp?x-image-process=image/format,png)
image.png
每个block最大存储128M,多出就会再多一个block去存储
文件在hdfs的实际路劲
[shaozhiqi@hadoop102 subdir0]$ ll
total 327200
-rw-rw-r--. 1 shaozhiqi shaozhiqi 50 Jun 27 21:18 blk_1073741825 -rw-rw-r--. 1 shaozhiqi shaozhiqi 11 Jun 27 21:18 blk_1073741825_1001.meta -rw-rw-r--. 1 shaozhiqi shaozhiqi 134217728 Jun 27 21:25 blk_1073741826 -rw-rw-r--. 1 shaozhiqi shaozhiqi 1048583 Jun 27 21:25 blk_1073741826_1002.meta -rw-rw-r--. 1 shaozhiqi shaozhiqi 134217728 Jun 27 21:26 blk_1073741827 -rw-rw-r--. 1 shaozhiqi shaozhiqi 1048583 Jun 27 21:26 blk_1073741827_1003.meta -rw-rw-r--. 1 shaozhiqi shaozhiqi 63998133 Jun 27 21:26 blk_1073741828 -rw-rw-r--. 1 shaozhiqi shaozhiqi 499995 Jun 27 21:26 blk_1073741828_1004.meta [shaozhiqi@hadoop102 subdir0]$ pwd /opt/module/hadoop-3.1.2/data/tmp/dfs/data/current/BP-1884196904-192.168.1.102-1561640654781/current/finalized/subdir0/subdir0 [shaozhiqi@hadoop102 subdir0]$ cat blk_1073741825 #可以确定整个就是我们的wc.input hadoop yarn hadoop mapreduce shaozhiqi shaozhiqi [shaozhiqi@hadoop102 subdir0]$
删除我们刚刚上传到hdfs的文件
[shaozhiqi@hadoop102 hadoop-3.1.2]$ bin/hdfs dfs -rm -r /wc.input
Deleted /wc.input
[shaozhiqi@hadoop102 hadoop-3.1.2]$ bin/hdfs dfs -rm -r /hadoop-3.1.2.tar.gz Deleted /hadoop-3.1.2.tar.gz [shaozhiqi@hadoop102 hadoop-3.1.2]$
验证是否删除,验证ok
![image.png](https://i-blog.csdnimg.cn/blog_migrate/dee6c4649c6544f6908e3f494c7d9d08.webp?x-image-process=image/format,png)
image.png