spark 节点缓存_如何使用简单的缓存策略优化节点请求

最新推荐文章于 2024-08-26 16:32:13 发布

culiu9261

最新推荐文章于 2024-08-26 16:32:13 发布

阅读量176

点赞数

文章标签：数据库 python java redis 分布式

原文链接：https://scotch.io/tutorials/how-to-optimize-node-requests-with-simple-caching-strategies

版权

spark 节点缓存

One of the things that affect how users interact with our applications is its speed. Even though some users generally have a poor connection, they are expecting some minimum level of speed when using the application. To give our users a seamless experience, we are going to consider the possibility of optimizing our requests in node by using the concept of caching.

影响用户如何与我们的应用程序交互的因素之一是它的速度。即使某些用户通常连接状况较差，但他们期望使用该应用程序时达到最低速度。为了给我们的用户带来无缝的体验，我们将考虑使用缓存的概念来优化节点中的请求的可能性。

Trust me, your users will thank you.

相信我，您的用户将感谢您。

In this article, we are going to look at how to optimize requests in our Node applications by caching responses to requests. Now, you may be wondering what caching is all about. Caching generally involves storing and retrieving data from a high-performance memory

在本文中，我们将研究如何通过缓存对请求的响应来优化Node应用程序中的请求。现在，您可能想知道什么是缓存。缓存通常涉及从高性能内存中存储和检索数据

为什么要使用缓存 ( Why Use A Cache )

The goal of caching is to save your server the stress of having to do the same thing over and over again that results in the same output. Let’s take for example, you have a page that generates a report of a companies inventory for the previous day for accountability. Now, let’s imagine that the report page will be viewed at least a thousand times daily by different auditors. Generating a single report wouldn’t be a problem, but if the same report is going to be generated every single time a thousand times, then you need to look for a better way to do it and that’s where caching comes in.

缓存的目的是减轻服务器的压力，即必须一遍又一遍地执行相同的操作，从而得到相同的输出。例如，您有一个页面可以生成前一天的公司库存报告以进行问责。现在，让我们想象一下，报告页每天将由不同的审核员至少浏览一千次。生成单个报告不是问题，但是如果将同一时间每千次生成相同的报告，那么您需要寻找一种更好的方法来进行缓存。

要求 ( Requirements )

To enable you to follow through the article, you need to have the following :

为了使您能够通读本文，您需要具备以下条件：

Node installed on your machine,
安装在您机器上的节点，
Node Package Manager ( NPM ) installed on your machine
您计算机上安装的Node Package Manager（ NPM ）
Some Basic Javascript Knowledge
一些基本JavaScript知识

To confirm your installations, run the following in your terminal :

要确认安装，请在终端中运行以下命令：

node --versionnpm --version

If you get version numbers as result, then you’re good to go.

如果得到版本号，那就很好了。

建立基本库存应用 ( Building A Basic Inventory Application )

We need to make a simple inventory reporting application to test caching capabilities with. The main goal is to look at the different approaches to caching in our node applications.

我们需要创建一个简单的库存报告应用程序来测试缓存功能。主要目标是研究节点应用程序中缓存的不同方法。

Since you already have npm installed on your machine, create a new project folder and install the necessary packages :

由于您已经在计算机上安装了npm ，请创建一个新的项目文件夹并安装必要的软件包：

mkdir inventory-app
    cd inventory-app 
    npm init && npm install --save express memory-cache flat-cache redis sqlite3

Let’s take a look at the index.js.

让我们看一下index.js 。

Import express and sqlite:

进口快递和直通车：

// index.js

    const express = require('express')
    const sqlite3 = require('sqlite3').verbose();
    const PORT = process.env.PORT || 3128 ;
    const app = express();

    [...]

express is a routing framework for Node which simplifies HTTP task. sqlite3 will power the database for this app. The application port is also specified and an express application is created.

express是Node的路由框架，可简化HTTP任务。 sqlite3将为该应用程序的数据库供电。还指定了应用程序端口并创建了快速应用程序。

Next thing we need to do is to create the server route. Just one route that points to a /products endpoint:

接下来，我们需要创建服务器路由。仅有一条路线指向/products端点：

// index.js
    [...]

    app.get('/products',  function(req, res){
        setTimeout(() => {
          let db = new sqlite3.Database('./NodeInventory.db');
          let sql = `SELECT * FROM products`;

          db.all(sql, [], (err, rows) => {
              if (err) {
                  throw err;
              }
              db.close();
              res.send( rows );
          });
          // this was wrapped in a setTimeout function to intentionally simulate a slow 
          // request
        }, 3000);
    });

    [...]

When a request is made to the /products route, a connection is made to our sqlite database and then the query’s executed to fetch all our products. Once this is done, the connection is closed and the response is sent back to the user with all the products that exist in our inventory.

当对/products路由发出请求时，将与我们的sqlite数据库建立连接，然后执行查询以获取我们的所有产品。完成此操作后，将关闭连接，并将响应与库存中存在的所有产品一起发送给用户。

The sqlite file is in the Github repository for reference.

sqlite文件位于Github存储库中以供参考。

To start listening to requests on the port, add the following at the end:

要开始侦听端口上的请求，请在末尾添加以下内容：

// index.js
    [...]

    app.listen(PORT, function(){
        console.log(`App running on port ${PORT}`);
    });

    [...]

Head to Postman or the browser and visit the /products route to get a list of the items stored in the database:

前往邮递员或浏览器并访问/products路线以获取存储在数据库中的项目的列表：

First run without caching gave a response time of 3043ms

So far our single-route app works, but imagine having to do this over and over again which is what will be the case when our daily inventory is being reviewed over a hundred times. Let’s take a look at how to optimize requests by making use of some caching functions

到目前为止，我们的单路径应用程序可以正常工作，但请想象一下，必须一遍又一遍地执行此操作，这就是每天对我们的每日库存进行一百次审核时的情况。让我们看一下如何通过使用一些缓存功能来优化请求

不同的缓存选项 ( Different Caching Options )

Using Memory-Cache for caching ( In - Memory ) Earlier on, the memory-cache module was installed. Now, let’s look at how to use it.

使用 Memory-Cache 进行缓存（In-Memory）较早时，已安装了memory-cache模块。现在，让我们看看如何使用它。

Import the module:

导入模块：

// index.js
    [...]

    cache cache = require('memory-cache');

    [...]

Create and configure cache middleware :

创建和配置缓存中间件：

// index.js

    // configure cache middleware
    let memCache = new cache.Cache();
    let cacheMiddleware = (duration) => {
        return (req, res, next) => {
            let key =  '__express__' + req.originalUrl || req.url
            let cacheContent = memCache.get(key);
            if(cacheContent){
                res.send( cacheContent );
                return
            }else{
                res.sendResponse = res.send
                res.send = (body) => {
                    memCache.put(key,body,duration*1000);
                    res.sendResponse(body)
                }
                next()
            }
        }
    }

    // app routes
    [...]

The duration representing how long the values need to stored in the cache will be specified. In the middleware, a unique key based on the request url is generated and a check is made to see if there is already content stored for that key. If content exists, the data is sent back as the response without having to make the extra query to our database.

将指定表示值需要在缓存中存储多长时间的持续时间。在中间件中，将基于request url生成一个唯一key ，并进行检查以查看是否已经为该key存储了内容。如果内容存在，则将数据作为响应发送回，而无需对我们的数据库进行额外的查询。

Now, if there is no content in the cache for the particular key, the request is processed as usual and the result of the request is stored in our cache before the response is sent to the user.

现在，如果高速缓存中没有特定密钥的内容，则照常处理请求，并且在将响应发送给用户之前，请求的结果将存储在我们的高速缓存中。

To use the cacheMiddleware edit the index.js to look like this :

要使用cacheMiddleware编辑index.js如下所示：

// index.js

    app.get('/products', cacheMiddleware(30), function(req, res){
         [...]    
    });

Notice how specified the duration we want the response for the request to be cached for was specified

注意如何指定了我们希望缓存请求响应的持续时间

We just looked at in-memory method for caching responses. One of the downsides of this method is that once the server goes down, all cached content is lost.

我们只是研究了用于缓存响应的内存中方法。该方法的缺点之一是，一旦服务器关闭，所有缓存的内容都会丢失。

Returns Products with a response time of 3045ms

Now, let's take a look at the using a caching method that persists the data on the server even after it has been restarted.

现在，让我们看一下使用缓存方法，即使重新启动数据后，该方法仍将数据保留在服务器上。

Using Flat file for caching We are going to now take a look at how to use files to persist our cached data on the server. To do this, we are going to make use of this package - flat-cache

使用平面文件进行缓存我们现在将看一下如何使用文件将缓存的数据持久保存在服务器上。为此，我们将使用此程序包- 平面缓存

Install the module by running :

通过运行以下命令安装模块：

npm install --save flat-cache

Now, tweak the index.js to make use of our new module. The index.js will look like this :

现在，调整index.js以使用我们的新模块。 index.js将如下所示：

// index.js
    [...]

    const flatCache = require('flat-cache')

    [...]

Then the new cache is loaded

然后加载新的缓存

// index.js 
    [...]

    // load new cache
    let cache = flatCache.load('productsCache');
    // optionally, you can go ahead and pass the directory you want your
    // cache to be loaded from by using this
    // let cache = flatCache.load('productsCache', path.resolve('./path/to/folder')

    [...]

Now that the cache has been loaded, the next thing to do now is to edit our application route so :

现在缓存已加载，接下来要做的是编辑我们的应用程序路由，如下所示：

// index.js
    [...]

    // create flat cache routes
    let flatCacheMiddleware = (req,res, next) => {
        let key =  '__express__' + req.originalUrl || req.url
        let cacheContent = cache.getKey(key);
        if( cacheContent){
            res.send( cacheContent );
        }else{
            res.sendResponse = res.send
            res.send = (body) => {
                cache.setKey(key,body);
                cache.save();
                res.sendResponse(body)
            }
            next()
        }
    };

    // create app routes 

    [...]

The logic behind this is similar to the in-memory cache. Check if a response already exists for our request using the key and it there already exists a response, it is returned to the user else, the request is executed and the response is saved to the cache before going to the user.

其背后的逻辑类似于内存中的缓存。使用key检查对我们的请求的响应是否已经存在，是否已经存在响应，然后将其返回给用户，否则将执行该请求并将响应保存到缓存中，然后再转到用户。

Notice that we ran the cache.save() . Doing this specifies that the cache is saved as a file in the directory we specified - if you specified any. If no directory was specified the module will determine where to save the ‘cache-file’

注意，我们运行了cache.save() 。这样做指定将缓存另存为文件，该文件位于我们指定的目录中（如果已指定）。如果未指定目录，则模块将确定将“缓存文件”保存在何处

To use the flatCacheMiddleware , edit the application routes so :

要使用flatCacheMiddleware ，请编辑应用程序路由，如下所示：

// index.js
    [...]

    // create app routes

    app.get('/products', flatCacheMiddleware, function(req, res){
      [...]
    });

Now, when the /products route is visited, the following result is obtained :

现在，当访问/products路由时，将获得以下结果：

Products are returned, with a response time of 3047ms

A productsCache file is also created with the following content :

还会创建具有以下内容的productsCache文件：

{"__express__/products":"[{\"ID\":1,\"NAME\":\"Shoes\",\"COUNT\":20},{\"ID\":2,\"NAME\":\"Bags\",\"COUNT\":12},{\"ID\":3,\"NAME\":\"Tables\",\"COUNT\":50},{\"ID\":5,\"NAME\":\"Laptop\",\"COUNT\":140},{\"ID\":6,\"NAME\":\"Chair\",\"COUNT\":200},{\"ID\":7,\"NAME\":\"Mouse\",\"COUNT\":20},{\"ID\":8,\"NAME\":\"Pen\",\"COUNT\":1000},{\"ID\":9,\"NAME\":\"Chips\",\"COUNT\":6303},{\"ID\":10,\"NAME\":\"Slides\",\"COUNT\":22},{\"ID\":11,\"NAME\":\"Console\",\"COUNT\":32}]"}

Using MemCached ( A Service ) Another option to consider for caching is memcached. Memcached is a caching client built for node JS with scaling in mind.

使用MemCached（一种服务）另一个考虑进行缓存的选项是memcached 。 Memcached是为节点JS构建的缓存客户端，考虑了扩展性。

To use the Memcached node client, you need to have memcached installed on your machine. Head over here to get it installed. When you have it installed, you then install the memcached node client by running :

要使用Memcached节点客户端，您需要在计算机上安装memcached 。前往这里进行安装。安装完毕后，您可以运行以下命令来安装memcached节点客户端：

npm install --save memcached

Now that the client is installed, import the module in the index.js:

现在已经安装了客户端，将模块导入index.js ：

// index.js 
    [...]

    const Memcached = require('memcached');

    [...]

Now, go ahead and configure the memcachedMiddleware by tweaking the index.js to look as follows :

现在，通过调整index.js使其外观如下，继续配置memcachedMiddleware ：

// index.js
    [...]

    let memcached = new Memcached("127.0.0.1:11211")

    let memcachedMiddleware = (duration) => {
        return  (req,res,next) => {
        let key = "__express__" + req.originalUrl || req.url;
        memcached.get(key, function(err,data){
            if(data){
                res.send(data);
                return;
            }else{
                res.sendResponse = res.send;
                res.send = (body) => {
                    memcached.set(key, body, (duration*60), function(err){
                        // 
                    });
                    res.sendResponse(body);
                }
                next();
            }
        });
    }
    };

    [...]

A memcached object is created and the object is connected to the PORT our Memcached instance is running on - in this case, 11211. Afterwards, a memcachedMiddleware is also created which is similar to the other middlewares created earlier. If there is no content for the specified key in the cache, the request is completed as usual and the response is stored in our cache. If there is a response for the specified key, the content is obtained from the cache and returned to the user without having to process the request.

创建了一个memcached对象，并将该对象连接到我们的Memcached实例正在运行的端口-在这种情况下为11211 。之后，还将创建一个memcachedMiddleware ，它与之前创建的其他中间件相似。如果缓存中没有指定键的内容，则请求将照常完成，并将响应存储在我们的缓存中。如果对指定的键有响应，则从缓存中获取内容，然后将其返回给用户，而不必处理该请求。

Now, edit the /products route and update it to use the memcachedMiddleware :

现在，编辑/products路由并更新它以使用memcachedMiddleware ：

// index.js
    [...]

    app.get("/products", memcachedMiddleware(20), function(req, res) {
      [...]
    });

    [...]

The memcachedMiddleware(20) above, specifies the duration ( in minutes ) for how long the key is to be stored in the cache.

上面的memcachedMiddleware(20)指定key在缓存中存储的持续时间（以分钟为单位）。

When a request is made to the /products route from Postman, the following result is obtained :

从Postman请求/products路由时，将获得以下结果：

Returns the products with a response time of 3049ms

Using Redis for caching Now, we have seen how to use in-memory cache and also how to persist our caches using files. Let’s take a look at using redis for caching in our application. Redis stands for **RE**``mote **D**``ictionary **S**``erver that has the ability to store and manipulate high-level data types.

使用Redis进行缓存现在，我们已经了解了如何使用in-memory缓存以及如何使用文件持久化缓存。让我们看一下在我们的应用程序中使用redis进行缓存。 Redis的代表**RE**``mote **D**``ictionary **S**``erver具有的能力来存储和操纵的高级别数据类型。

To get started using redis for caching application, install the redis node client by running :

要开始使用redis缓存应用程序，请运行以下命令安装redis节点客户端：

npm install -save redis

We also need to have a redis server running on our local machine. You can head over here to find out how.

我们还需要在本地计算机上运行一个redis服务器。您可以前往此处了解操作方法。

When you are sure your redis server is installed and working properly, the next thing to do is to import the necessary modules and create your redis client:

当您确定您的Redis服务器已安装并且可以正常工作时，下一步是导入必要的模块并创建您的Redis客户端：

// index.js
    [...]

    const redis = require('redis')
    const client = redis.createClient();

    [...]

After this, create a redisMiddleware and that’ll look like this :

之后，创建一个redisMiddleware ，它将如下所示：

// index.js
    [...]

    // create redis middleware
    let redisMiddleware = (req, res, next) => {
      let key = "__expIress__" + req.originalUrl || req.url;
      client.get(key, function(err, reply){
        if(reply){
            res.send(reply);
        }else{
            res.sendResponse = res.send;
            res.send = (body) => {
                client.set(key, JSON.stringify(body));
                res.sendResponse(body);
            }
            next();
        }
      });
    };

    // app routes
    [...]

A check is made to see if a response already exists for our request. If a response exists, then we fetch the value from the cache and return the response. If no response exists in the cache, then the request is executed the response is stored in the redis cache for future requests.

然后进行检查，看是否有response已经存在我们的要求。如果存在响应，则我们从缓存中获取值并返回响应。如果缓存中不存在响应，则将执行该请求，该响应将存储在redis缓存中以供将来的请求使用。

The /products route is also updated to use the redisMiddleware :

/products路由也已更新为使用redisMiddleware ：

// index.js 
    [...]

    app.get("/products", redisMiddleware, function(req, res) {
      [...]
    });

    // set port
    [...]

When a request to the /products server, you get :

当请求到/products服务器时，您将获得：

结果与建议 ( Results and Recommendation )

Now, we are going to compare performance for the different approaches we have taken in this article :

现在，我们将比较本文采用的不同方法的性能：

No Caching at all
完全不缓存
In-Memory Caching
内存中缓存
Caching to File
缓存到文件
Using Memcached
使用Memcached
Using Redis for Caching
使用Redis进行缓存

No Caching at All
根本不缓存

Run	Response Time
Ist Run	3043ms
2nd Run	3027ms
3rd Run	3019ms
4th Run	3009ms
Avg Time	3024.5ms

跑	响应时间
初跑	3043毫秒
第二次跑	3027ms
第三轮	3019ms
第四轮	3009毫秒
平均时间	3024.5毫秒

In-Memory Caching
内存中缓存

Run	Response Time
Ist Run	3045ms
2nd Run	23ms
3rd Run	4ms
4th Run	17ms
Avg Time	772.25ms

跑	响应时间
初跑	3045ms
第二次跑	23毫秒
第三轮	4毫秒
第四轮	17毫秒
平均时间	772.25毫秒

Caching To File
缓存到文件

Run	Response Time
Ist Run	3047ms
2nd Run	31ms
3rd Run	8ms
4th Run	13ms
Avg Time	774.75ms

跑	响应时间
初跑	3047ms
第二次跑	31毫秒
第三轮	8毫秒
第四轮	13毫秒
平均时间	774.75毫秒

Using MemCached
使用MemCached

Run	Response Time
Ist Run	3049ms
2nd Run	9ms
3rd Run	10ms
4th Run	16ms
Avg Time	771ms

跑	响应时间
初跑	3049ms
第二次跑	9毫秒
第三轮	10毫秒
第四轮	16毫秒
平均时间	771毫秒

Using Redis for Caching
使用Redis进行缓存

Run	Response Time
Ist Run	3049ms
2nd Run	6ms
3rd Run	18ms
4th Run	10ms
Avg Time	770.75ms

跑	响应时间
初跑	3049ms
第二次跑	6毫秒
第三轮	18毫秒
第四轮	10毫秒
平均时间	770.75毫秒

From the results above, we can see that deciding not caching the files at all does not help improve the response time. The different caching options we have looked at so far have proven to be successful in reducing the average load time.

从上面的结果中，我们可以看到决定完全不缓存文件无助于缩短响应时间。到目前为止，我们已经研究了不同的缓存选项，它们在减少平均加载时间方面被证明是成功的。

结论 ( Conclusion )

Caching comes in handy a lot of times, but you need to know when to use it. it may be an overkill especially when the requests made to the application aren’t frequent. Also, POST, PUT and DELETE methods should never be cached because you want unique resources to be affected each time a request is made.

缓存通常会派上用场，但是您需要知道何时使用它。这可能是一个矫kill过正，尤其是当对应用程序的请求不频繁时。另外，永远不要缓存POST ， PUT和DELETE方法，因为您希望每次发出请求时都会影响唯一资源。

Here’s a link to the GitHub repository. Feel free to leave your comments and questions below

这是GitHub存储库的链接。随时在下面留下您的评论和问题

翻译自: https://scotch.io/tutorials/how-to-optimize-node-requests-with-simple-caching-strategies

spark 节点缓存

culiu9261

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
spark 节点缓存_如何使用简单的缓存策略优化节点请求

spark 节点缓存One of the things that affect how users interact with our applications is its speed. Even though some users generally have a poor connection, they are expecting some minimum level of speed w...
复制链接

扫一扫