通过python在内存中对bytes数据进行压缩和解压

场景描述

在数据采集获取到数据后,就要考虑数据存储问题。

最近碰到一个比较有特点的网站,网站的背景图片是直接返回了base64编码展示在源码中,一个页面原始数据保存下来达到了近2M。因为项目要求,需要大量采集并存储,所以尝试对采集到的信息先在内存中进行压缩后存储。

压缩、解压源码

import io
import gzip

class InMemoryCompressor:
    def __init__(self):
        pass
    
    def compress(self,buff:bytes):


        buffer = io.BytesIO()
        with gzip.GzipFile(fileobj=buffer, mode='w') as gz_file:
            gz_file.write(buff)

        compressed_data = buffer.getvalue()
        return compressed_data

    def decompress(self, compressed_data: bytes):
        decompressed_buffer = io.BytesIO(compressed_data)
        with gzip.GzipFile(fileobj=decompressed_buffer, mode='r') as gz_file:
            decompressed_text = gz_file.read()

        return decompressed_text
  • 2
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
根据提供的引用内容,JPEG算法是一种有损压缩算法,而且实现起来比较复杂。Python有一些第三方库可以实现JPEG压缩解压缩,比如Pillow和OpenCV。下面是使用Pillow库实现JPEG压缩解压缩的示例代码: 压缩文件: ```python from PIL import Image # 打开原始图像 img = Image.open('original.jpg') # 将图像转换为YCbCr颜色空间 img_ycbcr = img.convert('YCbCr') # 对每个颜色通道进行离散余弦变换(DCT) img_dct = img_ycbcr.copy() img_dct = img_dct.transform((img.width // 8, img.height // 8), Image.EXTENT, (8, 8), Image.DCT) # 量化DCT系数 quantization_table = [ [16, 11, 10, 16, 24, 40, 51, 61], [12, 12, 14, 19, 26, 58, 60, 55], [14, 13, 16, 24, 40, 57, 69, 56], [14, 17, 22, 29, 51, 87, 80, 62], [18, 22, 37, 56, 68, 109, 103, 77], [24, 35, 55, 64, 81, 104, 113, 92], [49, 64, 78, 87, 103, 121, 120, 101], [72, 92, 95, 98, 112, 100, 103, 99] ] img_quantized = img_dct.copy() for x in range(img_dct.width): for y in range(img_dct.height): img_quantized.putpixel((x, y), tuple([round(img_dct.getpixel((x, y))[i] / quantization_table[y % 8][x % 8]) for i in range(3)])) # 将量化后的DCT系数写入文件 with open('compressed.jpg', 'wb') as f: for y in range(img_quantized.height): for x in range(img_quantized.width): for i in range(3): f.write(bytes([img_quantized.getpixel((x, y))[i]])) ``` 解压文件: ```python from PIL import Image # 从文件读取量化后的DCT系数 with open('compressed.jpg', 'rb') as f: img_quantized = Image.new('YCbCr', (512, 512)) for y in range(img_quantized.height): for x in range(img_quantized.width): img_quantized.putpixel((x, y), tuple([int.from_bytes(f.read(1), byteorder='big') for i in range(3)])) # 反量化DCT系数 quantization_table = [ [16, 11, 10, 16, 24, 40, 51, 61], [12, 12, 14, 19, 26, 58, 60, 55], [14, 13, 16, 24, 40, 57, 69, 56], [14, 17, 22, 29, 51, 87, 80, 62], [18, 22, 37, 56, 68, 109, 103, 77], [24, 35, 55, 64, 81, 104, 113, 92], [49, 64, 78, 87, 103, 121, 120, 101], [72, 92, 95, 98, 112, 100, 103, 99] ] img_dct = img_quantized.copy() for x in range(img_quantized.width): for y in range(img_quantized.height): img_dct.putpixel((x, y), tuple([img_quantized.getpixel((x, y))[i] * quantization_table[y % 8][x % 8] for i in range(3)])) # 对每个颜色通道进行反离散余弦变换(IDCT) img_ycbcr = img_dct.copy() img_ycbcr = img_ycbcr.transform((img_dct.width * 8, img_dct.height * 8), Image.EXTENT, (8, 8), Image.IDCT) # 将图像转换回RGB颜色空间 img_rgb = img_ycbcr.convert('RGB') # 保存解压后的图像 img_rgb.save('uncompressed.jpg') ```

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值