如何使用Rclone在区域之间传输DigitalOcean空间

最新推荐文章于 2023-11-21 20:28:57 发布

cukw6666

最新推荐文章于 2023-11-21 20:28:57 发布

阅读量450

点赞数

文章标签： python linux java 大数据运维

原文链接：https://www.digitalocean.com/community/tutorials/how-to-transfer-digitalocean-spaces-between-regions-using-rclone

版权

介绍 (Introduction)

DigitalOcean Spaces is an object storage service designed to make it easy and cost effective to store and serve large amounts of data.

DigitalOcean Spaces是一种对象存储服务，旨在使存储和提供大量数据变得容易且具有成本效益。

In this guide, we will cover how to migrate data between Spaces regions, by using Rclone to transfer data between two Spaces. We will demonstrate how to install Rclone, the configuration settings needed to access multiple regions, and the commands that you can use to synchronize your files between regions and verify their integrity.

在本指南中，我们将介绍如何使用Rclone在两个Space之间传输数据，从而在Spaces区域之间迁移数据。我们将演示如何安装Rclone，访问多个区域所需的配置设置以及可用于在区域之间同步文件并验证其完整性的命令。

创建API密钥并查找空间端点信息 (Creating API Keys and Finding Spaces Endpoint Information)

Before we begin installing and configuring Rclone to copy our objects between Spaces, we will need some information about our DigitalOcean Spaces account. We will need a Spaces API key, and we will need to know the regions and names of our source and destination Spaces.

在开始安装和配置Rclone在Space之间复制对象之前，我们需要有关DigitalOcean Spaces帐户的一些信息。我们将需要一个Spaces API密钥，并且我们需要知道源空间和目标空间的区域和名称。

生成DigitalOcean Spaces API密钥 (Generating a DigitalOcean Spaces API Key)

To create a DigitalOcean Spaces API key, follow the “Creating an Access Key” section of our How To Create a DigitalOcean Space API Key documentation.

要创建DigitalOcean Spaces API密钥，请遵循我们的如何创建DigitalOcean Space API密钥文档中的“创建访问密钥”部分。

Save the access key ID and the secret key. We will use them later to configure Rclone to access our account.

保存访问密钥ID和秘密密钥。我们稍后将使用它们来配置Rclone以访问我们的帐户。

查找与S3兼容的Spacepoints (Finding the Spaces S3-Compatible Endpoint)

Next, we need to find the endpoint for each Space. You can view the Space’s endpoint within the DigitalOcean Control Panel by selecting the Space and viewing the Settings tab:

接下来，我们需要找到每个Space的端点。您可以在DigitalOcean控制面板中通过选择Space并查看Settings标签来查看Space的端点：

The endpoint will always be the region you created the Space in, followed by .digitaloceanspaces.com. Make note of the endpoint for both of your Spaces. Will we use this information when creating our rclone configuration.

端点将始终是您在其中创建Space的区域，其后是.digitaloceanspaces.com 。记下两个Space的端点。创建rclone配置时，我们将使用此信息rclone ？

安装Rclone (Installing Rclone)

You’re now ready to install Rclone. You can do this on your local machine, or – if you are bandwidth limited – you may want to install Rclone on a Droplet located in the source or destination Spaces region.

现在您可以安装Rclone了。您可以在本地计算机上执行此操作，或者-如果您受带宽限制，则可能需要在源空间或目标空间区域中的Droplet上安装Rclone。

Visit the Downloads section of the project’s website to find binaries of the utility compiled for different platforms. Download the zipped binary that matches your computer’s operating system to get started.

访问项目网站的“ 下载”部分，以查找针对不同平台编译的实用程序的二进制文件。下载与您的计算机操作系统匹配的压缩二进制文件以开始使用。

Once you have the Rclone zip file downloaded to your computer, follow the section below that matches your platform.

将Rclone zip文件下载到计算机后，请按照下面与您的平台匹配的部分进行操作。

的Linux (Linux)

Before we can extract the archive, we will need to ensure that the unzip utility is available.

在提取档案之前，我们需要确保unzip实用程序可用。

If you are running Ubuntu or Debian, you can update the local package index and install unzip by typing:

如果您正在运行Ubuntu或Debian ，则可以输入以下内容来更新本地软件包索引并安装unzip ：

sudo apt update
sudo apt更新
sudo apt install unzip
sudo apt安装解压缩

If you are running CentOS or Fedora, you can install unzip by typing:

如果您正在运行CentOS或Fedora ，则可以通过输入以下命令安装unzip ：

sudo yum install unzip
sudo yum安装解压缩

With unzip installed, navigate to the directory where you downloaded the rclone zip file:

安装unzip ，导航到您下载rclone zip文件的目录：

cd ~/Downloads
cd〜 /下载

Next, unzip the archive and move into the newly created directory:

接下来，解压缩归档文件并移至新创建的目录：

unzip rclone*
解压缩rclone *
cd rclone-v*
cd rclone-v *

From here, we can copy the binary to the /usr/local/bin directory so that it is available system-wide:

从这里，我们可以将二进制文件复制到/usr/local/bin目录，以便在整个系统范围内可用：

sudo cp rclone /usr/local/bin
须藤cp rclone / usr / local / bin

Next, we add the manual page to our system, so that we can easily get help on the command syntax and available options. Make sure that the local manual directory is available and then copy the rclone.1 file:

接下来，我们将手册页添加到我们的系统中，以便我们可以轻松地获得有关命令语法和可用选项的帮助。确保本地手册目录可用，然后复制rclone.1文件：

sudo mkdir -p /usr/local/share/man/man1
须藤mkdir -p / usr / local / share / man / man1
sudo cp rclone.1 /usr/local/share/man/man1
须藤cp rclone.1 / usr / local / share / man / man1

Then update the man database to add the new manual page to the system:

然后更新man数据库以将新的手册页添加到系统中：

sudo mandb
须藤

Finally, we can create the Rclone configuration directory and open up a configuration file:

最后，我们可以创建Rclone配置目录并打开一个配置文件：

mkdir -p ~/.config/rclone
mkdir -p〜/ .config / rclone
nano ~/.config/rclone/rclone.conf
纳米〜/ .config / rclone / rclone.conf

This will open your text editor with a new blank file. Skip ahead to the section on Configuring Rclone to continue.

这将使用新的空白文件打开您的文本编辑器。跳至“ 配置Rclone ”部分以继续。

苹果系统 (macOS)

If you are running macOS, begin by navigating in the terminal to the directory where you downloaded the rclone zip file:

如果您正在运行macOS，请从终端中导航到您下载rclone zip文件的目录：

cd ~/Downloads
cd〜 /下载

Next, unzip the file and move into the newly created directory:

接下来，解压缩文件并移至新创建的目录：

unzip -a rclone*
解压缩-rclone *
cd rclone-v*
cd rclone-v *

Next, make sure the /usr/local/bin directory is available and then copy the rclone binary to it:

接下来，确保/usr/local/bin目录可用，然后将rclone二进制文件复制到该目录：

sudo mkdir -p /usr/local/bin
须藤mkdir -p / usr / local / bin
sudo cp rclone /usr/local/bin
须藤cp rclone / usr / local / bin

Finally, we can create the configuration directory and open up a configuration file:

最后，我们可以创建配置目录并打开一个配置文件：

mkdir -p ~/.config/rclone
mkdir -p〜/ .config / rclone
nano ~/.config/rclone/rclone.conf
纳米〜/ .config / rclone / rclone.conf

This will open up your text editor with a new blank file. Skip ahead to the section on Configuring Rclone to continue.

这将使用新的空白文件打开您的文本编辑器。跳至“ 配置Rclone ”部分以继续。

视窗 (Windows)

If you are running Windows, begin by navigating to the Downloads directory in the Windows File Explorer. Select the rclone zip file and right-click. In the context menu that appears, click Extract All…:

如果您正在运行Windows，请首先导航至Windows File Explorer中的Downloads目录。选择rclone zip文件，然后单击鼠标右键。在出现的上下文菜单中，单击全部提取… ：

Follow the prompts to extract the files from the zip archive.

按照提示从zip存档中提取文件。

The rclone.exe utility must be run from the command line. Open a new Command Prompt (the cmd.exe program) window by clicking the Windows button in the lower-left corner, typing cmd, and selecting Command Prompt.

rclone.exe实用程序必须从命令行运行。通过单击左下角的Windows按钮，键入cmd ，然后选择“ 命令提示符” ，以打开一个新的命令提示符 ( cmd.exe程序)窗口。

Inside, navigate to the rclone path you extracted by typing:

在内部，通过键入以下内容导航到您提取的rclone路径：

cd "%HOMEPATH%\Downloads\rclone*\rclone*"
cd“％HOMEPATH％\ Downloads \ rclone * \ rclone *”

List the directory contents to verify that you are in the correct location:

列出目录内容以验证您是否位于正确的位置：


   
   
    
    Output
   
   10/23/2017  01:02 PM    <DIR>          .
10/23/2017  01:02 PM    <DIR>          ..
10/23/2017  01:02 PM                17 git-log.txt
10/23/2017  01:02 PM           296,086 rclone.1
10/23/2017  01:02 PM        16,840,192 rclone.exe
10/23/2017  01:02 PM           315,539 README.html
10/23/2017  01:02 PM           261,497 README.txt
               5 File(s)     17,713,331 bytes
               2 Dir(s)  183,296,266,240 bytes free

You will need to be in this directory whenever you want to use the rclone.exe command.

每当您想使用rclone.exe命令时，都将需要位于该目录中。

Note: On macOS and Linux, we run the tool by typing rclone, but on Windows, the command is called rclone.exe. Throughout the rest of this guide, we will be providing commands as rclone, so be sure to substitute rclone.exe each time when running on Windows.

注意：在macOS和Linux上，我们通过键入rclone运行该工具，但在Windows上，该命令称为rclone.exe 。在本指南的其余部分中，我们将以rclone形式提供命令，因此请确保每次在Windows上运行时都替换rclone.exe 。

Next, we can create the configuration directory and open up a configuration file to define our S3 and Spaces credentials:

接下来，我们可以创建配置目录并打开一个配置文件以定义我们的S3和Spaces凭证：

mkdir "%HOMEPATH%\.config\rclone"
mkdir“％HOMEPATH％\。config \ rclone”
notepad "%HOMEPATH%\.config\rclone\rclone.conf"
记事本“％HOMEPATH％\。config \ rclone \ rclone.conf”

This will open up your text editor with a new blank file. Continue ahead to learn how to define your Spaces regions in the configuration file.

这将使用新的空白文件打开您的文本编辑器。继续继续学习以了解如何在配置文件中定义您的Spaces区域。

配置Rclone (Configuring Rclone)

We will configure our two DigitalOcean Spaces regions as Rclone “remotes” in the Rclone configuration file. Paste the following section in the configuration file to define the first region:

我们将在Rclone配置文件中将两个DigitalOcean Spaces区域配置为Rclone“远程”。将以下部分粘贴到配置文件中以定义第一个区域：

~/.config/rclone/rclone.conf

〜/ .config / rclone / rclone.conf

[spaces-sfo2]
type = s3
env_auth = false
access_key_id = your_spaces_access_key
secret_access_key = your_spaces_secret_key
endpoint = sfo2.digitaloceanspaces.com
acl = private

Here, we define a new rclone “remote” named spaces-sfo2. Change the region name to match the Spaces region you are configuring.

在这里，我们定义了一个新的名为spaces-sfo2 rclone “远程”。更改区域名称以匹配您正在配置的“空间”区域。

We set the type to s3 so that rclone knows the appropriate way to interact with and manage the remote storage resource. We will define the Spaces access credentials in this configuration file, so we can set env_auth to false.

我们将type设置为s3以便rclone知道与远程存储资源进行交互和管理的适当方法。我们将在此配置文件中定义Spaces访问凭据，因此我们可以将env_auth设置为false 。

Next, we set the access_key_id and secret_access_key variables to our Spaces access key and secret key, respectively. Be sure to change the values to the credentials associated with your account.

接下来，我们分别将access_key_id和secret_access_key变量设置为Spaces访问密钥和秘密密钥。确保将值更改为与您的帐户关联的凭据。

We set the endpoint to the Spaces endpoint we looked up earlier.

我们将endpoint设置为我们之前查找的Spaces端点。

Finally, we set the acl to private to protect our assets until we want to share them.

最后，我们将acl设置为private以保护我们的资产，直到我们想要共享它们为止。

Next, make a duplicate of the configuration block you just created, then update the name and endpoint region to reflect your second region:

接下来，复制刚创建的配置块，然后更新名称和端点区域以反映您的第二个区域：

~/.config/rclone/rclone.conf

〜/ .config / rclone / rclone.conf

. . .

[spaces-nyc3]
type = s3
env_auth = false
access_key_id = your_spaces_access_key
secret_access_key = your_spaces_secret_key
endpoint = nyc3.digitaloceanspaces.com
acl = private

The rest of the configuration should remain the same as for the first region. Save and close the file when you are finished.

其余配置应与第一个区域相同。完成后保存并关闭文件。

On macOS and Linux, be sure to lock down the permissions of the configuration file since our credentials are inside:

在macOS和Linux上，请确保锁定配置文件的权限，因为我们的凭据位于其中：

chmod 600 ~/.config/rclone/rclone.conf
chmod 600〜/ .config / rclone / rclone.conf

On Windows, permissions are denied to non-administrative users unless explicitly granted, so we shouldn’t need to adjust access manually.

在Windows上，除非明确授予权限，否则将拒绝非管理用户的权限，因此我们不需要手动调整访问权限。

Next, we’ll use rclone to explore our Spaces and sync data between them.

接下来，我们将使用rclone探索我们的空间并在它们之间同步数据。

将对象从S3复制到空间 (Copying Objects from S3 to Spaces)

Now that our configuration is complete, we are ready to transfer our files.

现在我们的配置已完成，我们可以传输文件了。

Begin by checking the rclone configured remotes:

首先检查rclone配置的遥控器：

rclone listremotes
rclone listremotes


   
   
    
    Output
   
   spaces-nyc3:
spaces-sfo2:

Both of the regions we defined are displayed.

显示我们定义的两个区域。

We can view the available Spaces by asking rclone to list the “directories” associated with the remotes (make sure to add the colon to the end of the remote name):

我们可以通过要求rclone列出与遥控器关联的“目录”来查看可用空间(确保在冒号名称的末尾添加冒号)：

rclone lsd spaces-sfo2:
rclone lsd spaces- sfo2 ：


   
   
    
    Output
   
             -1 2019-09-23 13:07:54        -1 source-space

The above output indicates that one Space, called source-space was found in the sfo2 region.

上面的输出表明在sfo2区域中找到了一个称为source-space 。

You can repeat the procedure to view the other region:

您可以重复此过程以查看其他区域：

rclone lsd spaces-nyc3:
rclone lsd空格-nyc3 ：


   
   
    
    Output
   
             -1 2019-09-23 13:08:28        -1 destination-space

To view the contents of a Space, you can use the tree command. Pass in the remote name, followed by a colon and the name of the “directory” you wish to list (the Space name):

要查看空间的内容，可以使用tree命令。输入远程名称，后接冒号和您要列出的“目录”的名称(空间名称)：

rclone tree spaces-sfo2:source-space
rclone树空间-sfo2 ：源空间


   
   
    
    Output
   
   /
├── Photos
│   ├── 2019.01.24-23.10.27.png
│   ├── 2019.01.24-23.11.39.png
│   ├── 2019.01.24-23.18.00.png
│   ├── 2019.01.24-23.18.18.png
│   ├── 2019.01.24-23.18.30.png
│   ├── 2019.01.24-23.19.32.png
│   ├── 2019.01.24-23.23.06.png
│   ├── 2019.01.24-23.23.53.png
│   ├── 2019.01.24-23.25.14.png
│   ├── 2019.01.24-23.26.22.png
│   ├── 2019.01.25-12.43.35.png
│   ├── 2019.03.13-14.35.34.png
│   └── 2019.03.13-14.40.52.png
└── Photos.zip

1 directories, 14 files

When you are ready, you can copy the files between Spaces by typing:

准备就绪后，可以通过键入以下内容在Spaces之间复制文件：

rclone sync spaces-sfo2:source-space spaces-nyc3:destination-space
rclone同步空间-sfo2 ：源空间空间 -nyc3 ：目标空间

Assuming everything went well, rclone will begin copying objects between the two Spaces.

假设一切顺利， rclone将开始在两个空间之间复制对象。

Note: If you hadn’t previously created the destination Space in the specified region, rclone will attempt to create one for you with the given name. This will fail if the name provided is already being used by another account or if the name doesn’t meet the naming requirements for DigitalOcean Spaces (lowercase letters, numbers, and dashes only).

注意：如果您以前未在指定区域中创建目标空间，则rclone将尝试使用给定名称为您创建一个目标空间。如果提供的名称已经被另一个帐户使用，或者该名称不满足DigitalOcean Spaces的命名要求(仅小写字母，数字和破折号)，则此操作将失败。

When the transfer is complete, you can check that the objects have transferred by viewing them with the tree subcommand:

传输完成后，可以使用tree子命令查看对象，以检查对象是否已传输：

rclone tree spaces-nyc3:destination-space
rclone树空间-nyc3 ：目标空间


   
   
    
    Output
   
   /
├── Photos
│   ├── 2019.01.24-23.10.27.png
│   ├── 2019.01.24-23.11.39.png
│   ├── 2019.01.24-23.18.00.png
│   ├── 2019.01.24-23.18.18.png
│   ├── 2019.01.24-23.18.30.png
│   ├── 2019.01.24-23.19.32.png
│   ├── 2019.01.24-23.23.06.png
│   ├── 2019.01.24-23.23.53.png
│   ├── 2019.01.24-23.25.14.png
│   ├── 2019.01.24-23.26.22.png
│   ├── 2019.01.25-12.43.35.png
│   ├── 2019.03.13-14.35.34.png
│   └── 2019.03.13-14.40.52.png
└── Photos.zip

1 directories, 14 files

For more robust verification, use the check subcommand to compare the objects in both regions:

要获得更可靠的验证，请使用check子命令比较两个区域中的对象：

rclone check spaces-sfo2:source-space spaces-nyc3:destination-space
rclone检查空间-sfo2 ：源空间空间 -nyc3 ：目标空间


   
   
    
    Output
   
   2019/09/23 14:29:11 NOTICE: S3 bucket destination-space: 0 differences found
2019/09/23 14:29:11 NOTICE: S3 bucket destination-space: 14 matching files

This will compare the hash values of each object in both remotes. You may receive a message indicating that some hashes could not be compared. In that case, you can rerun the command with the --size-only flag (which just compares based on file size) or the --download flag (which downloads each object from both remotes to compare locally) to verify the transfer integrity.

这将比较两个遥控器中每个对象的哈希值。您可能会收到一条消息，指出某些哈希无法比较。在这种情况下，您可以使用--size-only标志(仅根据文件大小进行比较)或--download标志(从两个远程--download下载每个对象以进行本地比较)重新运行该命令，以验证传输完整性。

结论 (Conclusion)

In this guide, we’ve covered how to transfer objects between two DigitalOcean Spaces regions. We gathered API credentials and endpoint information from the Spaces service, installed and configured the rclone utility on our local computer, and then copied all objects from a source Space to a destination Space.

在本指南中，我们介绍了如何在两个DigitalOcean Spaces区域之间传输对象。我们从Spaces服务收集了API凭据和终结点信息，在本地计算机上安装并配置了rclone实用程序，然后将所有对象从源Space复制到目标Space。

The rclone client can be used for many other object storage management tasks including uploading or downloading files, mounting Spaces on the local filesystem, and creating or deleting additional Spaces. Check out the man page to learn more about the functionality the tool provides.

rclone客户端可用于许多其他对象存储管理任务，包括上载或下载文件，在本地文件系统上安装空间以及创建或删除其他空间。查阅man页以了解有关该工具提供的功能的更多信息。

翻译自: https://www.digitalocean.com/community/tutorials/how-to-transfer-digitalocean-spaces-between-regions-using-rclone

cukw6666

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
如何使用Rclone在区域之间传输DigitalOcean空间

介绍 (Introduction)DigitalOcean Spaces is an object storage service designed to make it easy and cost effective to store and serve large amounts of data. DigitalOcean Spaces是一种对象存储服务，旨在使存储和提供大量数据变得容...
复制链接

扫一扫