rails 图片处理_如何使用Rails和Amazon S3以编程方式对用户图像进行后处理（包括测试）...

rails 图片处理

问题 (The problem)

In our platforms, we allow our users to upload their own images for profile pictures. This results, as you might imagine, in a wide variety of image sizing, quality and formats. We display these images in various ways throughout our platforms.

在我们的平台上，我们允许用户上传自己的图片作为个人资料图片。 您可能会想到，这会产生各种图像尺寸，质量和格式。 我们在整个平台上以各种方式显示这些图像。

For the most part, we can avoid sizing problems by manually setting sizing on the image tag. But in one very important place — emails — certain servers ignore our styling and display those images at full size: enormous.

在大多数情况下，我们可以通过在image标签上手动设置尺寸来避免尺寸问题。 但是在一个非常重要的地方-电子邮件中-某些服务器忽略了我们的样式，而是以全尺寸显示这些图像：巨大。

We need a way to reformat user images programmatically. Additionally, since we’re going to be messing with the images anyway, we’d like to auto-rotate, adjust levels and colors, and generally make them as nice and as consistent as we can.

我们需要一种以编程方式重新格式化用户图像的方法。 此外，由于无论如何我们都会弄乱图像，因此我们想自动旋转，调整色阶和颜色，并通常使它们尽可能的美观和一致。

注意事项 (Considerations)

• Our images are stored with Amazon’s S3 cloud storage. Fortunately Amazon offers a relatively easy-to-use API for interacting with their services.
  我们的图像存储在Amazon的S3云存储中。 幸运的是，亚马逊提供了一个相对易于使用的API来与其服务进行交互。
• Because our images are on S3, I thought it would be excellent to have this service as a Lambda function, triggered when a user uploads a photo. Unfortunately I could not, for the damn life of me, get anything to print in the CloudWatch console (where the logs should appear). After bashing up against this wall for a day, I decided to take it back in-house.
  因为我们的图像在S3上，所以我认为最好将此服务作为Lambda函数，并在用户上传照片时触发。 不幸的是，对于我该死的生活，我无法在CloudWatch控制台(应该在其中显示日志)中打印任何内容。 在墙上撞了一天之后，我决定将它带回内部。
• We host on Heroku, which offers a free and simple scheduler to run tasks. It’s not critical for us to have these images converted immediately upon upload. We can schedule a job that picks up everything new in the last 10 minutes, and convert it.
  我们在Heroku上托管，它提供了一个免费且简单的调度程序来运行任务。 对于我们来说，上载后立即转换这些图像并不重要。 我们可以安排一项工作，在最近10分钟内收集所有新内容并进行转换。

工人 (The Worker)

What’s needed now is a worker we can call as frequently as Heroku will allow us (10 minutes is the shortest interval).

现在需要的是我们可以按Heroku允许的频率(最短的间隔为10分钟)打电话给我们的工人。

聚集合适的用户 (Gathering the right users)

First we’ll gather all users that have images that need to be converted. We’ve been storing user images in a specific pattern in our S3 bucket that includes a files folder. We can just search for users whose profile pictures Regex matches in files:

首先，我们将收集所有需要转换图像的用户。 我们一直在以特定模式将用户图像存储在包含files夹的S3存储桶中。 我们只搜索那些个人档案图片Regex与files匹配的用户：

User.where(profilePictureUrl: { '$regex': %r(\/files\/) })

Your mileage may vary here, search-wise: we use a Mongo database.

在搜索方面，您的里程可能会有所不同：我们使用的是Mongo数据库。

Of course, we will be using a different pattern for processed images. This will only pick up those who have uploaded new images since the task last ran. We’ll loop through each of these users and perform the following.

当然，我们将对处理的图像使用不同的模式。 这只会选择自上次执行任务以来已上传新图像的用户。 我们将遍历所有这些用户并执行以下操作。

设置一个临时文件 (Setting up a temporary file)

We’ll need somewhere to store the image data we are going to manipulate. We can do that with a tmp folder. We’ll use this as a holding place for the image we want to upload to the new S3 location. We’ll name it as we’d like our final image to be named. We wanted to simplify and standardize images in our system, so we’re using the unique user id as the image name:

我们需要在某个地方存储要处理的图像数据。 我们可以使用tmp文件夹来实现。 我们将其用作存放要上传到新S3位置的图像的地方。 我们将其命名为我们希望命名最终图像的名称。 我们想要简化和标准化系统中的图像，因此我们使用唯一的用户ID作为图像名称：

@temp_file_location = "./tmp/#{user.id}.png"

获取原始图像并将其保存在本地 (Getting the raw image and saving it locally)

Now we’ll talk to our S3 bucket and get the user’s raw, giant, unformatted image:

现在，我们将讨论我们的S3存储桶，并获取用户的原始，巨大且未经格式化的图像：

key = URI.parse(user.profilePictureUrl).path.gsub(%r(\A\/), '')
s3 = Aws::S3::Client.new
response = s3.get_object(bucket: ENV['AWS_BUCKET'], key: key)

The key code there is taking the URL string that we’ve saved as the user’s profilePictureUrl and chopping off everything that’s not the end path to the picture.

那里的key代码采用了我们保存为用户的profilePictureUrl的URL字符串，并切断了不是图片最终路径的所有内容。

For instance, http://images.someimages.com/whatever/12345/image.png would return whatever/12345/image.png from that code. That’s exactly what S3 wants from us to find the image in our bucket. Here’s the handy aws-sdk gem working for us with get_object.

例如， http://images.someimages.com/whatever/12345/image.png whatever/12345/image.png将从该代码返回whatever/12345/image.png 。 这正是S3希望我们在存储桶中找到图像的原因。 这是与get_object配合使用的便捷aws-sdk宝石。

Now we can call response.body.read to get a blob of an image (blob is the right word, though it’s above my pay grade to really understand how images are sent back-and-forth across the web). We can write that blob locally in our tmp folder:

现在，我们可以调用response.body.read来获取图像的斑点(blob是正确的词，尽管它确实高于我的薪水等级，才能真正理解如何在网络上来回发送图像)。 我们可以在tmp文件夹中本地写入该blob：

File.open(@temp_file_location, 'wb') { |file| file.write(response.body.read) }

If we stop here, you’ll see you can actually open up that file in your temp folder (with the name you set above — in our case <user>.png ).

如果我们在这里停止，您会看到实际上可以在temp文件夹中打开该文件(使用上面设置的名称-在我们的情况下为<user> .png)。

处理图像 (Process the image)

Now we’ve got the image downloaded from Amazon, we can do whatever we want to it! ImageMagick is an amazing tool freely available for everybody.

现在我们已经从亚马逊下载了图像，我们可以做任何我们想做的事情！ ImageMagick是一个免费提供给所有人的神奇工具。

We used a pared-down version for Rails called MiniMagick. That gem also has a great API that makes things lickety-split easy. We don’t even have to do anything special to pick up the image. The @temp_file_location we used earlier to save the image will work fine to bring it to MiniMagick’s attention:

我们为Rails使用了简化版本的MiniMagick 。 该gem还具有出色的API，可以轻松解决问题。 我们甚至不需要做任何特别的事情来拾取图像。 我们之前用来保存图像的@temp_file_location可以很好地引起MiniMagick的注意：

image = MiniMagick::Image.new(@temp_file_location)

Here’s the settings for our photos, but there are tons of options to play with:

下面是我们的照片设置，但也有选项吨一起玩：

image.combine_options do |img|
  img.resize '300x300>'
  img.auto_orient
  img.auto_level
  img.auto_gamma
  img.sharpen '0x3'
  image.format 'png'
end

combine_options is a handy way to do a bunch of stuff to an image in one block. When it exits, the image is saved again where it was before. (Image formatting can’t be done with the img from combine_options.) Now that image file in our temporary folder is all kinds of post-processed!

combine_options是一种方便的方法，可以在一个块中对图像进行处理。 退出时，图像将再次保存到之前的位置。 (无法使用combine_options的img来完成图像格式化。)现在，我们临时文件夹中的图像文件已经过各种后处理！

上传回S3并另存为用户的新个人资料图片 (Upload back to S3 and save as the user’s new profile picture)

Now all we have to do is set up another connection to S3 and make the upload:

现在，我们要做的就是建立与S3的另一个连接并进行上传：

Aws.config.update(
  region: ENV['AWS_REGION'],
  credentials: Aws::Credentials.new(ENV['AWS_ACCESS_KEY_ID'], ENV['AWS_SECRET_ACCESS_KEY']))

s3 = Aws::S3::Resource.new
name = File.basename(@temp_file_location)
bucket = ENV['AWS_BUCKET'] + '-output'
obj = s3.bucket(bucket).object(name)
obj.upload_file(@temp_file_location, acl: 'public-read')

By convention with Lambda, auto tasks will send to a new bucket with the old bucket’s name plus “-output” appended, so I stuck with that. All formatted user images will be dumped into this bucket. Since we are naming the images by (unique) user ids, we are sure we’ll never overwrite one user’s picture with another.

按照Lambda的约定，自动任务将发送到新存储桶，并在旧存储桶的名称后加上“ -output”，因此我坚持使用。 所有格式化的用户图像都将转储到此存储桶中。 由于我们是用(唯一的)用户ID命名图像，因此我们确信我们永远不会用另一个用户的图片覆盖。

We create a new object with the new file’s name, in the bucket of our choice, and then we upload_file. It has to be public-read if we want it visible without a lot of headache on our clients (you may choose a different security option).

我们在选择的存储桶中使用新文件的名称创建一个新对象，然后我们upload_file 。 如果我们希望它可见而又不会给我们的客户带来很多麻烦，则必须public-read它(您可以选择其他安全选项)。

If that last line returns true (which it will, if the upload goes smoothly), we can update our user record:

如果最后一行返回true(如果上传顺利，它将返回true)，我们可以更新用户记录：

new_url = "https://s3.amazonaws.com/#{ENV['AWS_BUCKET']}-output/#{File.basename(@temp_file_location)}"
user.update(profilePictureUrl: new_url)

And that’s it! If we run this guy, we’ll auto-format and resize all user images in the system. All the original images will be in place in their old pattern (and in case anything goes wrong), and all users’ links will point to their new, formatted images.

就是这样！ 如果运行这个家伙，我们将自动格式化系统中的所有用户图像并调整其大小。 所有原始图像都将以旧模式放置(以防万一出现问题)，并且所有用户的链接都将指向其新的格式化图像。

测试中 (Testing)

We couldn’t possibly add a new feature to a Rails application without testing, right? Absolutely. Here’s what our tests for this look like:

如果不进行测试，就无法向Rails应用程序添加新功能，对吧？ 绝对。 这是我们对此的测试结果：

RSpec.describe Scripts::StandardizeImages, type: :service do
  let!(:user) { User.make!(:student, profilePictureUrl: 'https://s3.amazonaws.com/files/some_picture.jpg') }

  before do
    stub_request(:get, 'https://s3.amazonaws.com/files/some_picture.jpg')
      .with(
        headers: {
          'Accept' => '*/*',
          'Accept-Encoding' => 'gzip;q=1.0,deflate;q=0.6,identity;q=0.3',
          'Host' => 's3.amazonaws.com',
          'User-Agent' => 'Ruby'
        }
      )
      .to_return(status: 200, body: '', headers: {})
    allow_any_instance_of(MiniMagick::Image).to receive(:combine_options).and_return(true)
    allow_any_instance_of(Aws::S3::Object).to receive(:upload_file).and_return(true)
  end

  describe '.call' do
    it 'finds all users with non-updated profile pictures, downloads, reformats and then uploads new picture' do
      Scripts::StandardizeImages.call

      expect(user.reload.profilePictureUrl)
        .to eq "https://s3.amazonaws.com/#{ENV['AWS_BUCKET']}-output/#{user.to_param}.png"
    end
  end
end

If you look first at the test itself, you’ll see we are testing that our user’s new profile picture URL was saved correctly. The rest of it we don’t so much care about, since we don’t actually want our test downloading anything, and we probably don’t want to spend the time for our test to be manipulating images.

如果您先看一下测试本身，就会看到我们正在测试是否正确保存了用户的新个人资料图片URL。 其余的我们不太在意，因为我们实际上不希望测试下载任何内容，并且我们可能不想花时间让我们的测试处理图像。

But of course the code is going to try to talk to Amazon and spin up MiniMagick. Instead, we can stub those calls. Just in case this is new for you, I’ll run through this part.

但是，当然，该代码将尝试与Amazon交流并启动MiniMagick。 相反，我们可以对这些调用进行存根。 以防万一这对您来说是新的，我将遍历此部分。

存根电话 (Stubbing calls)

If you aren’t mocking calls in your tests, you probably ought to start doing that immediately. All that’s required is the Webmock gem. You require it in your rails_helper and that’s about it.

如果您不模拟测试中的调用，则可能应该立即开始这样做。 所需要的只是Webmock gem。 您在rails_helper需要它， rails_helper此而已。

When your test tries to make a call to an external source, you’ll get a message like this (I’ve hidden private keys and things with …s):

当您的测试尝试调用外部来源时，您会收到如下消息(我隐藏了私钥和带有... s的内容)：

WebMock::NetConnectNotAllowedError:
       Real HTTP connections are disabled. Unregistered request: GET https://...
You can stub this request with the following snippet:
stub_request(:get, "https://...").
         with(
           headers: {
          'Accept'=>'*/*',
          'Accept-Encoding'=>'',
          'Authorization'=>...}).
         to_return(status: 200, body: "", headers: {})

Just copy the stub_request bit and you’re well on your way to stubbing glory. You may need to return something in that body, depending on what you are doing with the external API call.

只需复制stub_request位，您就可以顺利获得荣耀。 您可能需要在该body返回某些内容，具体取决于您对外部API调用所做的事情。

I found it difficult to get this stubbed response to return something my code would see as an image, so I just stubbed the MiniMagick function as well. This works fine because we are not seeing the output in this test anyway. You’ll have to manually test that the image is getting the proper formatting.

我发现很难得到这样的响应以返回我的代码可以看到的图像，因此我也对MiniMagick函数也进行了MiniMagick 。 之所以能正常工作，是因为我们在此测试中始终看不到输出。 您必须手动测试图像是否获得正确的格式。

Alternatively, you can use Aws.config[:s3] = { stub_responses: true } in your test initializer or possibly on your rails_helper to stub all S3 requests.

另外，您可以在测试初始化​​程序中或可能在rails_helper上使用Aws.config[:s3] = { stub_responses: true }对所有S3请求进行存根。

最后一点：Travis CI (One final note: Travis CI)

Depending on what options you decide to apply to your image, you may find that Travis’ version of ImageMagick is not the same as yours. I tried lots of things to get Travis using the same ImageMagick as I was. In the end, I am stubbing the MiniMagick call, so it’s a moot point. But beware: if you don’t stub that function, you may find your CI failing because it doesn’t recognize a newer option (like intensity).

根据您决定应用于图像的选项，您可能会发现Travis的ImageMagick版本与您的版本不同。 我尝试了很多事情来使Travis使用与以前相同的ImageMagick。 最后，我对MiniMagick调用进行了处理，所以这是有争议的。 但要注意：如果您不使用该功能，则CI可能会失败，因为它无法识别更新的选项(例如intensity )。

Thanks for reading!

谢谢阅读！

翻译自: https://www.freecodecamp.org/news/how-to-post-process-user-images-programmatically-with-rails-amazon-s3-including-testing-c72645536b54/

rails 图片处理