node.js调用.c文件_在Node.js中分派S3文件-CSDN博客

node.js调用.c文件

Some of our intranet backends use S3 storage and GraphQL APIs. It’s a common scenario nowadays. This story is about how we deal with file attachments in our schemas and how our client code can get hold of the real files.

我们的某些Intranet后端使用S3存储和GraphQL API。如今，这是一种常见的情况。这个故事是关于我们如何处理模式中的文件附件以及我们的客户端代码如何获取实际文件。

具有基于JWT令牌的身份验证的简单GraphQL服务器 (A simple GraphQL server with JWT token based authentication)

Let’s start with a very simple server:

让我们从一个非常简单的服务器开始：

We’ll use Koa to build the http server and apollo-server-koa to integrate an Apollo GraphQL server with Koa.
我们将使用Koa构建http服务器，并使用apollo-server-koa将Apollo GraphQL服务器与Koa集成。
The GraphQL schema allows querying for the files of the logged user.
GraphQL模式允许查询已登录用户的文件。
It uses jsonwebtoken to authenticate users using JWT tokens. For the sake of simplicity it uses a password instead of a digital certificate to sign the tokens. We also assume that those tokens are generated elsewhere. The token payload always contains the user login.
它使用jsonwebtoken使用JWT令牌对用户进行身份验证。为了简单起见，它使用密码而不是数字证书来对令牌进行签名。我们还假设这些令牌是在其他位置生成的。令牌有效负载始终包含用户登录名。
To resolve the files we ask our S3 endpoint for all the entries in a test Bucket that match the following “path” files/<login>/**. We use the listObjectsV2 function from the Amazon aws-sdk package.
为了解析文件，我们向S3端点询问test 桶中与以下“路径” files/<login>/**相匹配的所有条目。我们使用Amazon aws-sdk包中的listObjectsV2函数。

If you don’t have access to an S3 account or you prefer to test all this stuff locally, you can use the fabulous minio. You can easily set up a local instance using docker and expose some folder through an S3 API compatible endpoint.

如果您无权使用S3帐户，或者希望在本地测试所有这些东西，则可以使用神话般的minio 。您可以使用docker轻松设置本地实例，并通过与S3 API兼容的端点公开某些文件夹。

I’m not digging further into any of these concepts. Each of them deserves a story of its own.

我不会进一步研究这些概念。他们每个人都有自己的故事。

Below you can see the tiniest implementation I can think of for this server:

在下面，您可以看到该服务器可以想到的最微小的实现：

With this in place we can make a GraphQL query in the playground at http://localhost:3000/graphql and get all the files.

有了这个，我们可以在操场上通过http：// localhost：3000 / graphql进行GraphQL查询，并获取所有文件。

Notice that, as I said before, you’ll need to get a token and set the Authorization header in the playground. You can use the following script to create tokens for testing purposes: new-api-token user1.

请注意，正如我之前所说，您将需要获取令牌并在操场上设置Authorization标头。您可以使用以下脚本来创建令牌以进行测试： new-api-token user1 。

Our GraphQL ouput is not very useful yet. The url fields don’t contain real endpoints. Let’s fix that.

我们的GraphQL输出不是很有用。 url字段不包含真实的端点。让我们修复它。

获取真实文件 (Getting the real files)

We’ll change the resolver return statement to return a collection of S3 signed urls using the getSignedUrlPromise method. A signed url points to the S3 storage server and includes a temporary access token to control how and when a client can access a resource.

我们将更改解析程序的return语句，以使用getSignedUrlPromise方法返回S3签名的url的集合。签名的url指向S3存储服务器，并包含一个临时访问令牌，以控制客户端如何以及何时访问资源。

Now we can visit the url in the url field returned by the GraphQL and download the file attachment.

现在，我们可以在GraphQL返回的url字段中访问url并下载文件附件。

This approach has several problems though:

但是，这种方法有几个问题：

We need to orchestrate and wait for several async calls to getSignedUrlPromise. Thus having a performance penalty (n+1 operations).
我们需要编排并等待对getSignedUrlPromise几个异步调用。因此具有性能损失(n + 1次操作)。
We don’t really know if the client is going to really request any of the files, so, preloading the signed urls could be a waste of time.
我们真的不知道客户端是否会真正请求任何文件，因此，预加载签名的URL可能会浪费时间。
The signed urls have expiration times (60 seconds in the example). We could make them non expirable but for security reasons we may don’t want to. Preloading all the urls could make them stale before the client requests the files.
签名的URL具有有效时间(在示例中为60秒)。我们可以使它们不过期，但出于安全原因，我们可能不希望这样做。在客户端请求文件之前，预加载所有URL可能会使它们过时。

To solve this problems we are going to dispatch the files on demand.

为了解决这个问题，我们将按需分发文件。

调度文件 (Dispatching files)

Let’s start by changing our resolver again to make the urls point to a new dispatch REST endpoint.

让我们从再次更改解析器开始，以使url指向新的调度REST端点。

Now we need to code the endpoint. We could do two things:

现在我们需要对端点进行编码。我们可以做两件事：

Read the data from the s3 bucket and stream it to the client.
从s3存储桶中读取数据，并将其流式传输到客户端。
Generate a signed url and redirect the client.
生成签名的URL并重定向客户端。

Let’s go with the second option. There’s no need to do ourselves what the S3 endpoint can do for us.

让我们来看第二个选项。无需自己做S3端点可以为我们做的事情。

We need to add the REST endpoint before we setup the GraphQL server to intercept the requests before they reach the Apollo middleware. The dispatcher route matches any Bucket and Bucket Key. It authenticates the user via the JWT token and then generates the signed url and redirects. The token must be added as a token url param to the dispatch url by the client.

在设置GraphQL服务器以拦截请求到达Apollo中间件之前，我们需要添加REST端点。调度程序路由匹配任何存储桶和存储桶密钥。它通过JWT令牌对用户进行身份验证，然后生成签名的url并重定向。客户端必须将令牌作为token URL参数添加到调度URL。

There’s a missing piece though. We are just checking that the token is valid. We are not doing any authorization at all. Using a user2 valid token we can access user1 files 😱.

不过有一块失踪的东西。我们只是在检查令牌是否有效。我们根本没有做任何授权。使用一个user2有效令牌，我们可以访问user1文件😱。

改善安全性 (Improving security)

Fear not. We can add authorization policies to our dispatcher.

不要怕。我们可以向我们的调度员添加授权策略。

Given that the dispatcher is pretty generic and matches any Bucket and Key, we can implement a regitry of checkers. Each checker must say if it must be applied to a request (the matcher function) and check the user permissions (the check function).

鉴于调度程序非常通用，并且与任何Bucket和Key匹配，因此我们可以实现检查程序的功能。每个检查者必须说出是否必须将其应用于请求( matcher功能)，并检查用户权限( check功能)。

We’ll traverse this registry to find the appropriate check for each request. As our last check we’ll have a default one that forbids any access.

我们将遍历此注册表以找到每个请求的适当检查。作为我们的最后检查，我们将使用默认值禁止任何访问。

We add this logic to the dispatcher swapping lines 8–9 with the following:

我们将此逻辑添加到调度程序的第8–9行交换中，内容如下：

And that’s it for this story: dispatching s3 files on demand with JWT based authentication/authorization. You can check the full server in all its big-one-file glory in this gist.

就是这样：通过基于JWT的身份验证/授权按需分发s3文件。您可以在本要点中查看完整服务器的所有大文件大荣耀。

加起来 (Summing up)

When dealing with S3 file references in a GraphQL schema it’s better to use an intermediate dispatcher to generate S3 signed url on demand and add authorization policies. This logic also applies to other kind of APIs, not only to GraphQL.

在GraphQL模式中处理S3文件引用时，最好使用中间调度程序来按需生成S3签名的url并添加授权策略。此逻辑也适用于其他类型的API，不仅适用于GraphQL。

In this story we’ve used a very straightforward implementation with coarse and naive error checking, and tons of hardcoded stuff. To put this in production will require more finesse 😅. I think it will still help you to grasp the concept.

在这个故事中，我们使用了非常简单的实现，包括粗略的天真错误检查以及大量的硬编码内容。将其投入生产将需要更多技巧。我认为它仍然可以帮助您掌握概念。

It’s quite easy to do all this stuff in node with npm packages like koa, apollo-server-koa, jsonwebtoken and aws-sdk.

使用npm包(例如koa ， apollo-server-koa ， jsonwebtoken和aws-sdk在节点中完成所有这些工作非常容易。

And yes, I think so too. The the aws-sdk fixation on PascalCase parameter names is obnoxious 😑.

是的，我也这样认为。 PascalCase参数名称的aws-sdk固定令人讨厌。