【Hadoop】HDFS操作文档指令集

HDFS常用指令

---

几乎与shell命令一致

• 常用指令

hdfs dfs -ls /
hdfs dfs -put
hdfs dfs -copyFromLocal
hdfs dfs -moveFromLocal
hdfs dfs -cat
hdfs dfs -text
hdfs dfs -get
hdfs dfs -mkdir 
hdfs dfs -mv  移动/改名
hdfs dfs -getmerge
hdfs dfs -rm
hdfs dfs -rmdir
hdfs dfs -rm -r

• 案例
  1. 创建文件夹

[imooc@imooc ~]$ hdfs dfs -ls /user/imooc
drwxr-xr-x   - imooc supergroup          0 2023-02-19 01:54 /user/imooc/.sparkStaging
drwxr-xr-x   - imooc supergroup          0 2023-02-17 20:12 /user/imooc/df.csv
drwxr-xr-x   - imooc supergroup          0 2023-02-19 01:27 /user/imooc/df.parquet
-rw-r--r--   1 imooc supergroup        223 2023-02-19 01:51 /user/imooc/job.yaml
-rw-r--r--   1 imooc supergroup        177 2023-02-19 01:39 /user/imooc/qdb.yaml
[imooc@imooc ~]$ hdfs dfs -mkdir /user/imooc/test_tmp
[imooc@imooc ~]$ hdfs dfs -ls /user/imooc
drwxr-xr-x   - imooc supergroup          0 2023-02-19 01:54 /user/imooc/.sparkStaging
drwxr-xr-x   - imooc supergroup          0 2023-02-17 20:12 /user/imooc/df.csv
drwxr-xr-x   - imooc supergroup          0 2023-02-19 01:27 /user/imooc/df.parquet
-rw-r--r--   1 imooc supergroup        223 2023-02-19 01:51 /user/imooc/job.yaml
-rw-r--r--   1 imooc supergroup        177 2023-02-19 01:39 /user/imooc/qdb.yaml
drwxr-xr-x   - imooc supergroup          0 2023-02-19 23:02 /user/imooc/test_tmp

2. 查看目录下的文件

[imooc@imooc ~]$ hdfs dfs -ls
drwxr-xr-x   - imooc supergroup          0 2023-02-19 01:54 .sparkStaging
drwxr-xr-x   - imooc supergroup          0 2023-02-17 20:12 df.csv
drwxr-xr-x   - imooc supergroup          0 2023-02-19 01:27 df.parquet
-rw-r--r--   1 imooc supergroup        223 2023-02-19 01:51 job.yaml
-rw-r--r--   1 imooc supergroup        177 2023-02-19 01:39 qdb.yaml

3. 查看文件夹总大小

[imooc@imooc ~]$ hdfs dfs -du -h /user
2023-02-19 23:04:18,279 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
390    390    /user/hive
2.4 G  2.4 G  /user/imooc

4. 查看文件内容

[imooc@imooc ~]$ hdfs dfs -cat /user/imooc/job.yaml
metrics:
  - /user/imooc/qdb.yaml
inputs:
output:
  jdbc:
    connectionUrl: "jdbc:mysql://192.168.72.134:3306/sjjc"
    user: "root"
    password: "*******"
    driver: "com.mysql.jdbc.Driver"
explain: true
showQuery: true

5. 移动文件

[imooc@imooc ~]$ hadoop fs -ls /user/imooc
Found 6 items
drwxr-xr-x   - imooc supergroup          0 2023-02-19 01:54 /user/imooc/.sparkStaging
drwxr-xr-x   - imooc supergroup          0 2023-02-17 20:12 /user/imooc/df.csv
drwxr-xr-x   - imooc supergroup          0 2023-02-19 01:27 /user/imooc/df.parquet
-rw-r--r--   1 imooc supergroup        223 2023-02-19 01:51 /user/imooc/job.yaml
-rw-r--r--   1 imooc supergroup        177 2023-02-19 01:39 /user/imooc/qdb.yaml
drwxr-xr-x   - imooc supergroup          0 2023-02-19 23:02 /user/imooc/test_tmp
[imooc@imooc ~]$ hadoop fs -mv /user/imooc/test_tmp /user
[imooc@imooc ~]$ hadoop fs -ls /user
Found 3 items
drwxr-xr-x   - imooc supergroup          0 2022-07-23 21:06 /user/hive
drwxr-xr-x   - imooc supergroup          0 2023-02-19 23:11 /user/imooc
drwxr-xr-x   - imooc supergroup          0 2023-02-19 23:02 /user/test_tmp

6. 删除文件

[imooc@imooc ~]$ hadoop fs -rm -r /user/test_tmp
Deleted /user/test_tmp
[imooc@imooc ~]$ hadoop fs -ls /user
Found 2 items
drwxr-xr-x   - imooc supergroup          0 2022-07-23 21:06 /user/hive
drwxr-xr-x   - imooc supergroup          0 2023-02-19 23:11 /user/imooc

7. 上传到hdfs系统

[imooc@imooc ~]$ hdfs dfs -put qdb.txt /user/imooc
[imooc@imooc ~]$ hadoop fs -ls /user/imooc
Found 6 items
drwxr-xr-x   - imooc supergroup          0 2023-02-19 01:54 /user/imooc/.sparkStaging
drwxr-xr-x   - imooc supergroup          0 2023-02-17 20:12 /user/imooc/df.csv
drwxr-xr-x   - imooc supergroup          0 2023-02-19 01:27 /user/imooc/df.parquet
-rw-r--r--   1 imooc supergroup        223 2023-02-19 01:51 /user/imooc/job.yaml
-rw-r--r--   1 imooc supergroup        384 2023-02-19 23:13 /user/imooc/qdb.txt
-rw-r--r--   1 imooc supergroup        177 2023-02-19 01:39 /user/imooc/qdb.yaml

8. 下载到本地

[imooc@imooc ~]$ hdfs dfs -get /user/imooc/qdb.txt /tmp
[imooc@imooc ~]$ ll /tmp |grep qdb
-rw-r--r--. 1 imooc imooc 384 2月  19 23:14 qdb.txt

---