图像压缩数据集_使用图像进行数据压缩

图像压缩数据集

by Dan Ruta

通过Dan Ruta

使用图像进行数据压缩 (Data compression using images)

The original motivation for this side-project was finding a better way to save and load weights trained by a browser based neural network, while developing jsNet. To save weights, as JSON, I would have to log the content out, in the console, or on the page, and as you can imagine, it got pretty bad when the networks got big and there was a lot of data.

该副项目的最初动机是在开发jsNet的同时，找到一种更好的方法来保存和加载由基于浏览器的神经网络训练的权重。 为了减轻重量，作为JSON，我必须在控制台或页面上注销内容，并且可以想象，当网络很大且有大量数据时，情况变得非常糟糕。

The solution I came up with was to encode the data as images, which are handled much better than plain JSON text. Inadvertently, the algorithm I used to do that seems to not only have worked pretty well, but the eventual file size was just as small as gzip compression, even actually beating it in most cases.

我想出的解决方案是将数据编码为图像，处理起来比纯JSON文本要好得多。 不经意间，我曾经使用的算法似乎不仅运行良好，而且最终文件的大小与gzip压缩一样小，甚至在大多数情况下都超过了它。

So I thought I’d polish and release it as a standalone library, for general use. This article serves as an overview of the compression algorithm, for those who are into that sort of thing.

所以我认为我会完善它并将其作为独立库发布，以供一般使用。 本文是对压缩算法的概述，适合那些从事此类工作的人员。

算法 (The Algorithm)

Without further ado, a brief, high level overview of the array to image conversion is as follows:

事不宜迟，下面简要概述了阵列到图像的转换：

• The numbers are converted to base15, with the first few characters being metadata
  数字将转换为base15，前几个字符是元数据
• The results are concatenated
  结果是串联的
• Every pair of hex characters is then converted to a Uint8Clamped value
  然后将每对十六进制字符转换为Uint8Clamped值
• Finally, this is drawn to a canvas, then either returned (browser), or saved to a file (nodejs)
  最后，将其绘制到画布上，然后返回(浏览器)或保存到文件(nodejs)

As for the conversion from an image back into array data:

至于从图像转换回数组数据：

• The image is read either from a file, a Uint8ClampedArray, or from an img HTML element
  从文件，Uint8ClampedArray或从img HTML元素读取图像
• Each value in the Uint8ClampedArray is converted to base 16
  Uint8ClampedArray中的每个值都将转换为基数16
• Once concatenated into a string, the output is split along the metadata “f” ch