Node.js Zlib模块

最新推荐文章于 2024-05-03 17:16:20 发布

福州司马懿

最新推荐文章于 2024-05-03 17:16:20 发布

阅读量1.2w

点赞数 4

分类专栏： Node.js 文章标签： node.js zlib

Node.js 专栏收录该内容

39 篇文章 1 订阅

订阅专栏

Zlib
- Compressing HTTP requests and responses
- Memory Usage Tuning
- Flushing
- Constants
- Class Options
- Class: zlib.Deflate
- Class: zlib.DeflateRaw
- Class: zlib.Gunzip
- Class: zlib.Gzip
- Class: zlib.Inflate
- Class: zlib.InflateRaw
- Class: zlib.Unzip
- Class: zlib.Zlib
  - zlib.flush([kind], callback)
  - zlib.params(level, strategy, callback)
  - zlib.reset()
- zlib.constants
- zlib.createDeflate([options])
- zlib.createDeflateRaw([options])
- zlib.createGunzip([options])
- zlib.createGzip([options])
- zlib.createInflate([options])
- zlib.createInflateRaw([options])
- zlib.createUnzip([options])
- Convenience Methods
  - zlib.deflate(buf[, options], callback)
  - zlib.deflateSync(buf[, options])
  - zlib.deflateRaw(buf[, options], callback)
  - zlib.deflateRawSync(buf[, options])
  - zlib.gunzip(buf[, options], callback)
  - zlib.gunzipSync(buf[, options])
  - zlib.gzip(buf[, options], callback)
  - zlib.gzipSync(buf[, options])
  - zlib.inflate(buf[, options], callback)
  - zlib.inflateSync(buf[, options])
  - zlib.inflateRaw(buf[, options], callback)
  - zlib.inflateRawSync(buf[, options])
  - zlib.unzip(buf[, options], callback)
  - zlib.unzipSync(buf[, options])

The zlib module provides compression functionality implemented using Gzip and Deflate/Inflate. It can be accessed using:

zlib模块提供了用Gzip和Deflate/Inflate实现的压缩功能。它可以通过如下方式访问：

const zlib = require('zlib');

Compressing or decompressing a stream (such as a file) can be accomplished by piping the source stream data through a zlib stream into a destination stream:

压缩或解压一个流（诸如一个文件）可以通过管道将源数据流通过一个zlib流转化为目标流。

const gzip = zlib.createGzip();
const fs = require('fs');
const rs = fs.createReadStream('input.txt');
const ws = fs.createWriteStream('input.txt.gz');
rs.pipe(gzip).pipe(ws);

这里写图片描述

It is also possible to compress or decompress data in a single step:

它也可以在一步里面压缩或解压数据：

const zlib = require('zlib');
const input = '.................................';
zlib.deflate(input, (err, buffer) => {
    if(err) {
        throw err;
    }
    console.log(buffer.toString('base64'));
    //输出 eJzT0yMAAGTvBe8=
});

zlib.deflateSync(input).toString('base64');
//返回 'eJzT0yMAAGTvBe8='

const buffer = Buffer.from('eJzT0yMAAGTvBe8=', 'base64');
zlib.unzip(buffer, (err, buffer) => {
    if(err) {
        throw err;
    }
    console.log(buffer.toString());
    //输出 .................................
});

zlib.unzipSync(buffer).toString();
//返回 '................................'

Compressing HTTP requests and responses

The zlib module can be used to implement support for the gzip and deflate content-encoding mechanisms defined by HTTP.

The HTTP Accept-Encoding header is used within an http request to identify the compression encodings accepted by the client. The Content-Encoding header is used to identify the compression encodings actually applied to a message.

Note: the examples given below are drastically simplified to show the basic concept. Using zlib encoding can be expensive, and the results ought to be cached. See Memory Usage Tuning for more information on the speed/memory/compression tradeoffs involved in zlib usage.

Running a gzip operation on every request is quite expensive. It would be much more efficient to cache the compressed buffer.

zlib模块可以用来实现对HTTP定义的gzip和deflate内容编码机制的支持。

HTTP支持编码类型（accept-Encoding）首部用在http请求中，来定义客户端支持的压缩编码类型。内容编码（content-encoding）头部用来定义消息实际应用的压缩编码类型。

注意: 下面的例子已经被大大的简化了，用来展示最基本的概念。使用zlib编码的代价是非常昂贵的，并且它的结果应该（ought to）被缓存。详细信息可以查看内存使用调整章节（Memory Usage Tuning），以在速度/内存/压缩中权衡利弊，来决定是否使用zlib。

每一次请求的时候都执行一遍gzip操作的代价是非常昂贵的。因此缓存压缩的数据是非常有意义的。

const zlib = require('zlib');
const http = require('http');
const fs = require('fs');

//client request example
const request = http.get( { host: 'example.com', 
                            path: '/',
                            port: 80,
                            headers: { 'Accept-Encoding': 'gzip,deflate' } } );
request.on('response', (response) => {
    var output = fs.createWriteStream('example.html');
    var contentEncoding = response.headers['content-encoding'];
    console.log('response.headers[\'content-encoding\']: ' + contentEncoding);
    switch(contentEncoding) {
        case 'gzip':
            response.pipe(zlib.createGunzip()).pipe(output);
            break;
        case 'deflate':
            response.pipe(zlib.createInflate(zlib.createInflate())).pipe(output);
            break;
        default:
            response.pipe(output);
            break;
    }
});

//server example
http.createServer((request, response) => {
    var rawStream = fs.createReadStream('example.html');
    var acceptEncoding = request.headers['accept-encoding'];
    console.log('response.headers[\'accept-encoding\']: ' + acceptEncoding);
    if(!acceptEncoding) {
        acceptEncoding = '';
    }
    if(acceptEncoding.match(/\bgzip\b/)) {
        response.writeHead(200, {'content-encoding': 'gzip'});
        rawStream.pipe(zlib.createGzip()).pipe(response);
    } else if(acceptEncoding.match(/\bdeflate\b/)) {
        response.writeHead(200, {'content-encoding': 'deflate'});
        rawStream.pipe(zlib.createDeflate).pipe(response);
    } else {
        response.writeHead(200, {});
        rawStream.pipe(response);
    }
}).listen(8080);

这里写图片描述

注意：在get请求中JSON对象headers中的键值对无论大小写均是合法的，因为服务器在将其转化成对象时，会自动将这些键值对全部转为小写。因此在从response.headers对象中去取属性的时候，只能用小写的键名去取。

By default, the zlib methods with throw an error when decompressing truncated data. However, if it is known that the data is incomplete, or the desire is to inspect only the beginning of a compressed file, it is possible to suppress the default error handling by changing the flushing method that is used to compressed the last chunk of input data:

默认的情况是：当zlib函数压缩的数据被截断的时候，将会抛出一个异常。然而，如果众所周知该数据是不完整的，或者期望只在压缩文件开始部分检查，可以通过改变冲刷函数来阻止默认的错误处理操作，该方法经常用在压缩最后一块输入数据的时候。

const zlib = require('zlib');
// This is a truncated version of the buffer from the above examples
//完整的应该是 'eJzT0yMAAGTvBe8='
const buffer = Buffer.from('eJzT0yMA', 'base64');

zlib.unzip(buffer, { finishFlush: zlib.Z_SYNC_FLUSH }, (err, buffer) => {
  if (!err) {
    console.log(buffer.toString());
  } else {
    // handle error
    console.error('error occurs');
  }
});
//输出 ..

zlib.unzip(buffer, (err, buffer) => {
  if (!err) {
    console.log(buffer.toString());
  } else {
    // handle error
    console.error('error occurs');
  }
});
//输出 error occurs

This will not change the behavior in other error-throwing situations, e.g. when the input data has an invalid format. Using this method, it will not be possible to determine whether the input ended prematurely or lacks the integrity checks, making it necessary to manually check that the decompressed result is valid.

该方法不会改变其他错误被抛出时的行为，举个例子，但输入数据是非法的格式的时候，使用该方法，它将无法判断出输入的数据已经结束了还是缺乏完整性校验，因此手动校验压缩结果的完整性是非常必要的。

Memory Usage Tuning

内存使用调整

From zlib/zconf.h, modified to node.js’s usage:

The memory requirements for deflate are (in bytes):
在zlib/zconf.h中，可以修改node.js的内存使用量：
deflate操作的内存需求量为（单位：字节）：

(1 << (windowBits+2)) +  (1 << (memLevel+9))

That is: 128K for windowBits=15 + 128K for memLevel = 8 (default values) plus a few kilobytes for small objects.

For example, to reduce the default memory requirements from 256K to 128K, the options should be set to:

( 1 << ( 15+2 ) + ( 1 << ( 8 + 9 ) ) = 128KB，默认值windowBits=15，memLevel=8，另外加上一些很小的对象的极少的几千字节的空间。

举个例子，将默认的内存使用量从256KB减少到128KB，其选项可以被这样设置：

{ windowBits: 14, memLevel: 7 }

This will, however, generally degrade compression.

The memory requirements for inflate are (in bytes)

然而这样通常需要压缩降级。

inflate操作的内存需求量为（单位：字节）

1 << windowBits

That is, 32K for windowBits=15 (default value) plus a few kilobytes for small objects.

This is in addition to a single internal output slab buffer of size chunkSize, which defaults to 16K.

1 << 15 = 32KB，默认值windowBits=15，另外加上一些很小的对象的极少的几千字节的空间。

除此之外，还有一个单独的比较大的内部输出缓冲区chunkSize，它的默认值是16KB。

The speed of zlib compression is affected most dramatically by the level setting. A higher level will result in better compression, but will take longer to complete. A lower level will result in less compression, but will be much faster.

In general, greater memory usage options will mean that Node.js has to make fewer calls to zlib because it will be able to process more data on each write operation. So, this is another factor that affects the speed, at the cost of memory usage.

等级的设置对zlib压缩速度的影响是非常引人注目的。较大的压缩率会造成较大的压缩量，但是将会花掉很长的时间去完成该操作。较小的压缩率会生成较小的压缩量，但是操作会更快。

通常，设置更大的内存量选项意味着Node.js很少会使用zlib，因为它可以在每一次的写操作中处理更多的数据。同样，另外一个影响速度的因素也是内存的使用量。

Flushing

冲刷操作

Calling .flush() on a compression stream will make zlib return as much output as currently possible. This may come at the cost of degraded compression quality, but can be useful when data needs to be available as soon as possible.

In the following example, flush() is used to write a compressed partial HTTP response to the client:

在压缩流中调用flush()方法将会使得zlib当前返回近可能多的输出数据。这将会以降低压缩品质为代价，但是当数据尽快要用的时候该方法是非常有效的。

从下面的例子可以看出，flush()函数可以用来写一个压缩的部分的HTTP响应到客户端（打开浏览器也可以正常接收数据，但中文是乱码）：

const zlib = require('zlib');
const http = require('http');
const process = require('process');
http.createServer((request, response) => {
    //For the sake of simplicity, the Accept-Encoding checks are omitted.
    //因为考虑到简单的缘故，因此省略Accept-Encoding的校验
    response.writeHead(200, {'content-encoding': 'gzip'});
    const gzip = zlib.createGzip();
    gzip.pipe(response);
    setInterval(()=>{
        gzip.write(`The current time is ${Date()}\n`, ()=>{
            //The data has been passed to zlib, but the compression algorithm may have decided to buffer the data for more efficient compression. Calling .flush() will make the data avaliable as soon as the client is ready to receive it.
            //数据已经被转化为zlib，但是压缩算法可能会将数据缓存起来使得压缩效率更高。调用.flush()方法会使得数据立马被压缩，一旦客户端准备好就可以随时接收数据。
            gzip.flush();
            console.log('writen');
        });
    }, 1000);
}).listen(8080);

const request = http.get( { host:'127.0.0.1',
                            path:'/',
                            port:8080,
                            headers: {'Accept-Encoding':'gzip'} } );
request.on('response', (response)=>{
    console.log('response');
    //response事件只会被触发一次，由于管道已经建立，因此服务器端的输出会源源不断地流向该管道，而不用反复触发response事件
    if(response.headers['content-encoding'] === 'gzip') {
        response.pipe(zlib.createGunzip()).pipe(process.stdout);
    } else {
        console.log('response.headers[\'content-encoding\']:', response.headers['content-encoding']);
    }
});

Constants

常量

Added in: v0.5.8

All of the constants defined in zlib.h are also defined on require(‘zlib’). In the normal course of operations, it will not be necessary to use these constants. They are documented so that their presence is not surprising. This section is taken almost directly from the zlib documentation. See http://zlib.net/manual.html#Constants for more details.

所有的常量都定义在zlib.h文件中，也定义在require(‘zlib’)对象中。在一个正常的操作中，是不需要使用这些常量的。它们被记录在案，因此他们的出现也不足为奇。这个章节大部分都是直接从zlib文档中搬过来的。详细信息请参阅 http://zlib.net/manual.html#Constants

Allowed flush values.

允许冲刷的值

zlib.Z_NO_FLUSH
zlib.Z_PARTIAL_FLUSH
zlib.Z_SYNC_FLUSH
zlib.Z_FULL_FLUSH
zlib.Z_FINISH
zlib.Z_BLOCK
zlib.Z_TREES

Return codes for the compression/decompression functions.

压缩和解压缩函数的返回码。

Negative values are errors, positive values are used for special but normal events.

负值表示错误，正值用来表示一些特殊的但是正常的事件。

zlib.Z_OK
zlib.Z_STREAM_END
zlib.Z_NEED_DICT
zlib.Z_ERRNO
zlib.Z_STREAM_ERROR
zlib.Z_DATA_ERROR
zlib.Z_MEM_ERROR
zlib.Z_BUF_ERROR
zlib.Z_VERSION_ERROR

Compression levels.

压缩等级

zlib.Z_NO_COMPRESSION
zlib.Z_BEST_SPEED（需要的话）
zlib.Z_BEST_COMPRESSION
zlib.Z_DEFAULT_COMPRESSION

Compression strategy.

压缩策略

zlib.Z_FILTERED
zlib.Z_HUFFMAN_ONLY（哈弗曼）
zlib.Z_RLE（run-length encoding游程编码，或行程长度编码，或变动长度编码法，在控制论中对于二值图像而言是一种编码方法）
zlib.Z_FIXED
zlib.Z_DEFAULT_STRATEGY

The deflate compression method (the only one supported in this version).

deflate压缩方法（只在该版本被支持）

zlib.Z_DEFLATED

For initializing zalloc, zfree, opaque.

用来透明的初始化、释放资源。

zlib.Z_NULL

Class Options

Added in: v0.11.1

Each class takes an options object. All options are optional.

每一个类都有一个参数对象。所有的参数都是可选的。

Note that some options are only relevant when compressing, and are ignored by the decompression classes.

注意：一些选项只与压缩有关，对于解压的类可以忽略。

flush (default: zlib.Z_NO_FLUSH)
finishFlush (default: zlib.Z_FINISH)
chunkSize (default: 16*1024)
windowBits
level (compression only)
memLevel (compression only)
strategy (compression only)
dictionary (deflate/inflate only, empty dictionary by default)

See the description of deflateInit2 and inflateInit2 at http://zlib.net/manual.html#Advanced for more information on these.

更多关于 deflateInit2 和 inflateInit2 的描述信息在如下地址 http://zlib.net/manual.html#Advanced

Class: zlib.Deflate

Added in: v0.5.8
Compress data using deflate.

使用deflate压缩数据。

Class: zlib.DeflateRaw

Added in: v0.5.8
Compress data using deflate, and do not append a zlib header.

使用dflate压缩数据，并且不添加zlib头部。

Class: zlib.Gunzip

Added in: v0.5.8
Decompress a gzip stream.

解压一个gzip流。

Class: zlib.Gzip

Added in: v0.5.8
Compress data using gzip.

使用gzip压缩数据。

Class: zlib.Inflate

Added in: v0.5.8
Decompress a deflate stream.

解压deflate流。

Class: zlib.InflateRaw

Added in: v0.5.8
Decompress a raw deflate stream.

解压一个原始（不带zlib头部）的deflate流。

Class: zlib.Unzip

Added in: v0.5.8
Decompress either a Gzip- or Deflate-compressed stream by auto-detecting the header.

通过自动检测头部，解压Gzip或Deflate压缩流。

Class: zlib.Zlib

Added in: v0.5.8
Not exported by the zlib module. It is documented here because it is the base class of the compressor/decompressor classes.

不是由zlib模块导出的。它之所以被归档在这里是因为它是压缩器/解压器的基类。

zlib.flush([kind], callback)

Added in: v0.5.8
kind defaults to zlib.Z_FULL_FLUSH.

Flush pending data. Don’t call this frivolously, premature flushes negatively impact the effectiveness of the compression algorithm.

Calling this only flushes data from the internal zlib state, and does not perform flushing of any kind on the streams level. Rather, it behaves like a normal call to .write(), i.e. it will be queued up behind other pending writes and will only produce output when data is being read from the stream.

kind参数默认是zlib.Z_FULL_FLUSH

冲刷待处理的数据。不要愚昧地调用它，过早的冲刷会对压缩算法的有效性产生负面的影响。

调用它只会在zlib的内部状态中被冲刷，并不会再流级别上执行冲刷。相反的，它的行为就像一个普通的写调用。它将会排在其它待写的操作后面，并且只有当数据正在被读的时候才会产生输出。

zlib.params(level, strategy, callback)

Added in: v0.11.4
Dynamically update the compression level and compression strategy. Only applicable to deflate algorithm.

动态更新压缩级别和压缩策略。只适用于deflate（压缩）算法。

zlib.reset()

Added in: v0.7.0
Reset the compressor/decompressor to factory defaults. Only applicable to the inflate and deflate algorithms.

重置压缩器/解压器为出厂设置。只有inflate（解压）和deflate（压缩）算法适用该函数。

zlib.constants

Provides an object enumerating Zlib-related constants.

返回一个枚举了Zlib相关常量的对象。

zlib.createDeflate(options)

Added in: v0.5.8
Returns a new Deflate object with an options.

返回一个新的带参数的Deflate对象（用于压缩）。

zlib.createDeflateRaw(options)

Added in: v0.5.8
Returns a new DeflateRaw object with an options.

返回一个新的带参数的DeflateRaw对象（生成的流不使用zlib头部，用于压缩）。

zlib.createGunzip(options)

Added in: v0.5.8
Returns a new Gunzip object with an options.

返回一个新的带参数的Gunzip对象（用于解压）。

zlib.createGzip(options)

Added in: v0.5.8
Returns a new Gzip object with an options.

返回一个新的带参数的Gzip对象（用于压缩）。

zlib.createInflate(options)

Added in: v0.5.8
Returns a new Inflate object with an options.

返回一个新的带参数的Infalte对象（用于解压）。

zlib.createInflateRaw(options)

Added in: v0.5.8
Returns a new InflateRaw object with an options.

返回一个新的带参数的InflateRaw对象（生成的流不使用zlib头部，用于解压）。

zlib.createUnzip(options)

Added in: v0.5.8
Returns a new Unzip object with an options.

返回一个新的带参数的Unzip对象（用于解压）。

Convenience Methods

便利的方法

All of these take a Buffer or string as the first argument, an optional second argument to supply options to the zlib classes and will call the supplied callback with callback(error, result).

Every method has a *Sync counterpart, which accept the same arguments, but without a callback.

所有的这些函数的第一个参数都是字节数组或者字符串
xxx
并且会调用提供的回调函数callback(error, result)。

每一个方法都有一个同步的副本，它支持的参数除回调函数外均相同。