【CSDN软件工程师能力认证学习精选】python 内存数据压缩为zip

 CSDN软件工程师能力认证（以下简称C系列认证）是由中国软件开发者网CSDN制定并推出的一个能力认证标准。C系列认证历经近一年的实际线下调研、考察、迭代、测试，并梳理出软件工程师开发过程中所需的各项技术技能，结合企业招聘需求和人才应聘痛点，基于公开、透明、公正的原则，甑别人才时确保真实业务场景、全部上机实操、所有过程留痕、存档不可篡改。

我们每天将都会精选CSDN站内技术文章供大家学习，帮助大家系统化学习IT技术。

工作中需要将大批的数据，压缩为zip存储。按照传统的处理办法需要将数据先存储到本地磁盘，再从磁盘读文件压缩成zip文件。
传统方法需要多次磁盘IO，性能很低，如果跳过文件存储，直接将内存的数据压缩保存，会大大减少磁盘IO，提升性能。

不需要看解析的，可以直接看最后完整的python代码

创建一个类: InMemoryZIP(), 来处理所有的程序。

class InMemoryZIP(object):

一 init() 创建一个类文件对象

	def __init__(self):
		# create the in-memory file-like object
		self.in_memory_zip = BytesIO()

二 append() 内存数据添加到zip对象

	def append(self, filename_in_zip, file_contents):
		""" Appends a file with name filename_in_zip \
        and contents of file_contents to the in-memory zip.
        """

        # create a handle to the in-memory zip in append mode\
		if not isinstance(file_contents, bytes):
			file_contens = bytes(str(file_contens), encoding='utf-8')

		# write the file to the in-memory zip
		zf = zipfile.ZipFile(self.in_memory_zip, 'a', zipfile.ZIP_DEFLATED, False)

		zf.writestr(filename_in_zip, file_contents)

		# mark the files as having been created on Windows
        # so that Unix permissions are not inferred as 0000
		for zfiel in zf.filelist:
			zfile.create_system = 0

		return self

三 appendfile() 文件添加到zip对象

	def appendfile(self, file_path, file_name=None):
		""" Read a file with path file_path \
        and append to in-memory zip with name file_name.
        """

        # file_path is abs path
		if file_name is None:
			file_name = os.path.split(file_path)[1]


		with open(file_path, 'rb') as f:
			file_contents = f.read()
			self.append(file_name, file_contents)

		return self

四 read() 读取zip数据流

	def read(self):
		""" Returns a string with the contents of the in-memory zip.
        """

		self.in_memory_zip.seek(0)
		return self.in_memory_zip.read()

五 writetofile() 内存zip流保存为zip文件

def writetofile(self, zip_filename):
	"""
        Write the in-memory zip to a file
    """

	with open(zip_filename, 'wb') as f:
		f.write(self.read())

六 完整版python代码

# !user/bin/env python3
# -*-coding : utf-8 -*-

import zipfile
from io import BytesIO
import os


class InMemoryZIP(object):
    def __init__(self):
        # create the in-memory file-like object
        self.in_memory_zip = BytesIO()

    def append(self, filename_in_zip, file_contents):
        """ Appends a file with name filename_in_zip \
        and contents of file_contents to the in-memory zip.
        """
        # create a handle to the in-memory zip in append mode\
        if not isinstance(file_contents, bytes):
            file_contents = bytes(str(file_contents), encoding='utf-8')

        zf = zipfile.ZipFile(self.in_memory_zip, 'a',
                             zipfile.ZIP_DEFLATED, False)

        # write the file to the in-memory zip
        zf.writestr(filename_in_zip, file_contents)

        # mark the files as having been created on Windows
        # so that Unix permissions are not inferred as 0000
        for zfile in zf.filelist:
            zfile.create_system = 0
        return self

    def appendfile(self, file_path, file_name=None):
        """ Read a file with path file_path \
        and append to in-memory zip with name file_name.
        """
        if file_name is None:
            file_name = os.path.split(file_path)[1]

        f = open(file_path, 'rb')
        file_contents = f.read()
        self.append(file_name, file_contents)
        f.close()
        return self

    def read(self):
        """ Returns a string with the contents of the in-memory zip.
        """
        self.in_memory_zip.seek(0)
        return self.in_memory_zip.read()

    def writetofile(self, filename):
        """
        Write the in-memory zip to a file
        """
        f = open(filename, 'wb')
        f.write(self.read())
        f.close()


if __name__ == '__main__':

	pass

	# contens = 'xxxxxxxxx'  # 内存数据
    # imz = InMemoryZIP()
    # imz.append('test.txt', contens)
    # imz.writetofile('test.zip')
   

关于CSDN软件工程师能力认证

      CSDN软件工程师能力认证（以下简称C系列认证）是由中国软件开发者网CSDN制定并推出的一个能力认证标准。C系列认证历经近一年的实际线下调研、考察、迭代、测试，并梳理出软件工程师开发过程中所需的各项技术技能，结合企业招聘需求和人才应聘痛点，基于公开、透明、公正的原则，甑别人才时确保真实业务场景、全部上机实操、所有过程留痕、存档不可篡改。C系列认证的宗旨是让一流的技术人才凭真才实学进大厂拿高薪，同时为企业节约大量招聘与培养成本，使命是提升高校大学生的技术能力，为行业提供人才储备，为国家数字化战略贡献力量。

 

了解详情可点击：CSDN软件工程师能力认证介绍

 

本文出处：https://blog.csdn.net/littleRpl/article/details/110120178?ops_request_misc=%257B%2522request%255Fid%2522%253A%2522161422986216780269847200%2522%252C%2522scm%2522%253A%252220140713.130102334.pc%255Fall.%2522%257D&request_id=161422986216780269847200&biz_id=0&utm_medium=distribute.pc_search_result.none-task-blog-2~all~first_rank_v2~rank_v29-9-110120178.pc_search_result_before_js&utm_term=python+数据压缩&spm=1018.2226.3001.4187