hdfs命令_HDFS命令

hdfs命令

In this lesson on Apache Hadoop HDFS commands, we will go through the most common commands which are used for Hadoop administration and to manage files present on a Hadoop cluster.

在本课程中，有关Apache Hadoop HDFS命令的内容，我们将介绍最常用的命令，这些命令用于Hadoop管理和管理Hadoop集群上的文件。

HDFS命令 (HDFS Commands)

Hive commands can be run on any Hadoop cluster or you’re free to use any of the VMs offered by Hortonworks, Cloudera etc.

Hive命令可以在任何Hadoop群集上运行，或者您可以自由使用Hortonworks ， Cloudera等提供的任何VM。

In this guide, we will make use of Ubuntu 17.10 (GNU/Linux 4.13.0-37-generic x86_64) machine:

在本指南中，我们将使用Ubuntu 17.10（GNU / Linux 4.13.0-37-generic x86_64）计算机：

Ubuntu Version

Ubuntu版本

Finally, we will make use of Hadoop v3.0.1 for this lesson:

最后，本课将使用Hadoop v3.0.1：

Hadoop version

Hadoop版本

Let’s get started.

让我们开始吧。

Hadoop HDFS命令 (Hadoop HDFS Commands)

We will start with some very basic help commands and go into more detail as we go through this lesson.

我们将从一些非常基本的帮助命令开始，并在本课程中进行更详细的介绍。

获取所有HDFS命令 (Getting all HDFS Commands)

The simplest help command for Hadoop HDFS is the following with which we get all the available commands in Hadoop and how to use them:

以下是Hadoop HDFS最简单的帮助命令，通过它我们可以获取Hadoop中所有可用的命令以及如何使用它们：

hadoop fs -help

Let’s see the output for this command:

让我们看一下该命令的输出：

Hadoop fs help

Hadoop fs帮助

The output was quite long actually as this prints all the available commands a brief on how to use those commands as well.
实际上，输出相当长，因为这会打印所有可用的命令，并简要说明如何使用这些命令。

有关特定Hadoop命令的帮助 (Help on specific Hadoop command)

The information printed from the last command was quite big as it printed all the commands. Finding help for a specific command is tricky in that output. Here is a command to narrow your search:

从最后一个命令打印的信息很大，因为它打印了所有命令。 在该输出中查找特定命令的帮助非常棘手。 这是缩小搜索范围的命令：

hadoop fs -help ls

Let’s see the output of this command:

让我们看一下该命令的输出：

Hadoop specific command guide

Hadoop特定命令指南

特定Hadoop命令的用法 (Usage of specific Hadoop command)

to know the syntax of each command, we don’t need t go anywhere apart from the terminal itself. We can know the syntax of a command on how to use it, use the usage option:

要知道每个命令的语法，除了终端本身，我们不需要走任何地方。 我们可以使用用法选项来了解有关如何使用命令的语法：

hadoop fs -usage ls

Let’s see the output of this command:

让我们看一下该命令的输出：

Usage of Hadoop Command

Hadoop命令的用法

Apart from usage, it also shows all possible options for the command specified.
除用法外，它还显示指定命令的所有可能选项。

列出fs文件和目录 (Listing fs files and directories)

To list all the available files and subdirectories under default directory, just use the following command:

要列出默认目录下的所有可用文件和子目录，只需使用以下命令：

hadoop fs -ls

Let’s see the output for this command:

让我们看一下该命令的输出：

Listing all files

列出所有文件

We ran this in the root directory and that’s why the output.
我们在根目录中运行它，这就是输出的原因。

制作HDFS目录 (Making HDFS Directory)

We can make a new directory for Hadoop File System using the following command:

我们可以使用以下命令为Hadoop File System创建新目录：

hadoop fs -mkdir /root/journaldev_bigdata

Note that if you create a new directory inside the /user/ directory, Hadoop will have read/write permissions on the directory but with other directories, it only has read permission by default.

请注意，如果在/user/目录中创建一个新目录，则Hadoop将对该目录具有读/写权限，但对于其他目录，默认情况下它仅具有读权限。

将文件从本地文件系统复制到Hadoop FS (Copying file from Local file System to Hadoop FS)

To copy a file from Local file System to Hadoop FS, we can use a simple command:

要将文件从本地文件系统复制到Hadoop FS，我们可以使用一个简单的命令：

hadoop fs -copyFromLocal derby.log /root/journaldev_bigdata

Let’s see the output for this command:

让我们看一下该命令的输出：

Copy File from local fs to HDFS

将文件从本地fs复制到HDFS

If instead of copying the file, you just want to move it, just make use of the
如果您只想移动文件而不是复制文件，则只需使用 -moveFromLocal option. -moveFromLocal选项。

磁盘使用情况 (Disk Usage)

We can see the disk usage of files under HDFS in a given directory with a simple option as shown:

我们可以通过一个简单的选项查看给定目录中HDFS下文件的磁盘使用情况，如下所示：

hadoop fs -du /root/journaldev_bigdata/

Let’s see the output for this command:

让我们看一下该命令的输出：

Disk Usage of a directory in HDFS

HDFS中目录的磁盘使用情况

If you simply want to check disk usage of complete HDFS, run the following command:
如果您只想检查完整HDFS的磁盘使用情况，请运行以下命令：

Let’s see the output for this command:

让我们看一下该命令的输出：

Disk Usage of complete HDFS

完整HDFS的磁盘使用情况

清空垃圾数据 (Empty Trash Data)

When we are sure that no files in the trash are usable, we can empty the trash in HDFS by deleting all files with the following command:

当我们确定垃圾箱中没有可用的文件时，我们可以通过使用以下命令删除所有文件来清空HDFS中的垃圾箱：

hadoop fs -expunge

This will simply delete all Trashed data in the HDFS and creates no output.

这将仅删除HDFS中的所有Trashed数据，并且不创建任何输出。

修改文件的复制因子 (Modifying replication factor for a file)

As we already know, replication factor is the count by which a file is replicated across as Hadoop cluster and in its HDFS. We can modify the replication factor of a file using the following command:

众所周知，复制因子是文件在Hadoop集群及其HDFS中被复制的计数。 我们可以使用以下命令修改文件的复制因子：

hadoop fs -setrep -w 1 /root/journaldev_bigdata/derby.log

Let’s see the output of this command:

让我们看一下该命令的输出：

Modify replication factor in HDFS

修改HDFS中的复制因子

更新Hadoop目录权限 (Updating Hadoop Directory permissions)

If you face permission related issues in Hadoop, run the following command:

如果您在Hadoop中遇到与权限相关的问题，请运行以下命令：

hadoop fs -chmod 700 /root/journaldev_bigdata/

With this command, you can provide and formulate the permissions given to a HDFS directory and restrict its access.

使用此命令，您可以提供和制定授予HDFS目录的权限并限制其访问。

删除HDFS目录 (Removing HDFS Directory)

We can remove an entire HDFS directory using the rm command:

我们可以使用rm命令删除整个HDFS目录：

hadoop fs -rm -r /root/journaldev_bigdata

Let’s see the output for this command:

让我们看一下该命令的输出：

Removing directory from HDFS

从HDFS移除目录

That’s all for a quick roundup on Hadoop HDFS commands.

这就是对Hadoop HDFS命令的快速总结。

翻译自: https://www.journaldev.com/20624/hdfs-commands

hdfs命令