Python中msgpack库的使用

msgpack用起来像json,但是却比json快,并且序列化以后的数据长度更小,言外之意,使用msgpack不仅序列化和反序列化的速度快,数据传输量也比json格式小,msgpack同样支持多种语言。

安装

直接pip即可

pip install msgpack

使用

import msgpack

var = {'a': 'this',
       'b': 'is',
       'c': 'a test'
}

with open('data.txt', 'wb') as f1:
    msgpack.dump(var, f1) # 存储数据
with open('data.txt', 'rb') as f2:
    var = msgpack.load(f2, use_list=False, encoding='utf-8') # 读取数据
print(var)

输出结果

{'a': 'this', 'b': 'is', 'c': 'a test'}

存储类型

使用type函数查看上述代码中提取出的var的类型

print(type(var))

输出结果

<class 'dict'>

可知msgpack提取出的数据类型为dict,即其以字典形式储存

关于查看变量数据类型的方法可以参考Python中查看变量数据类型

与json效率对比

import json
import msgpack

stu = {
    'name': 'lili',
    'age': 18,
    'score': 100
}

# 序列化
msg_str = msgpack.packb(stu)
print(len(msg_str))
json_str = json.dumps(stu)
print(len(json_str))

# 反序列化
stu_dict = msgpack.unpackb(msg_str, use_list=False, encoding='utf-8')
print(stu_dict)

输出结果

23
41
{'name': 'lili', 'age': 18, 'score': 100}

程序的运行结果表明,msgpack序列化后的字符串长度为23,而json模块序列化后的字符串长度为41,接近节省了一半的空间。

参考资料

【python】msgpack使用
python序列化第2篇—msgpack

  • 4
    点赞
  • 15
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
======================= MessagePack for Python ======================= :author: INADA Naoki :version: 0.4.1 :date: 2014-02-17 .. image:: https://secure.travis-ci.org/msgpack/msgpack-python.png :target: https://travis-ci.org/#!/msgpack/msgpack-python What's this ------------ `MessagePack <http://msgpack.org/>`_ is a fast, compact binary serialization format, suitable for similar data to JSON. This package provides CPython bindings for reading and writing MessagePack data. Install --------- You can use ``pip`` or ``easy_install`` to install msgpack:: $ easy_install msgpack-python or $ pip install msgpack-python PyPy ^^^^^ msgpack-python provides pure python implementation. PyPy can use this. Windows ^^^^^^^ When you can't use binary distribution, you need to install Visual Studio or Windows SDK on Windows. (NOTE: Visual C++ Express 2010 doesn't support amd64. Windows SDK is recommanded way to build amd64 msgpack without any fee.) Without extension, using pure python implementation on CPython runs slowly. Notes ----- Note for msgpack 2.0 support ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ msgpack 2.0 adds two types: *bin* and *ext*. *raw* was bytes or string type like Python 2's ``str``. To distinguish string and bytes, msgpack 2.0 adds *bin*. It is non-string binary like Python 3's ``bytes``. To use *bin* type for packing ``bytes``, pass ``use_bin_type=True`` to packer argument. >>> import msgpack >>> packed = msgpack.packb([b'spam', u'egg'], use_bin_type=True) >>> msgpack.unpackb(packed, encoding='utf-8') ['spam', u'egg'] You shoud use it carefully. When you use ``use_bin_type=True``, packed binary can be unpacked by unpackers supporting msgpack-2.0. To use *ext* type, pass ``msgpack.ExtType`` object to packer. >>> import msgpack >>> packed = msgpack.packb(msgpack.ExtType(42, b'xyzzy')) >>> msgpack.unpackb(packed) ExtType(code=42, data='xyzzy') You can use it with ``default`` and ``ext_hook``. See below. Note for msgpack 0.2.x users ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The msgpack 0.3 have some incompatible changes. The default value of ``use_list`` keyword argument is ``True`` from 0.3. You should pass the argument explicitly for backward compatibility. `Unpacker.unpack()` and some unpack methods now raises `OutOfData` instead of `StopIteration`. `StopIteration` is used for iterator protocol only. How to use ----------- One-shot pack & unpack ^^^^^^^^^^^^^^^^^^^^^^ Use ``packb`` for packing and ``unpackb`` for unpacking. msgpack provides ``dumps`` and ``loads`` as alias for compatibility with ``json`` and ``pickle``. ``pack`` and ``dump`` packs to file-like object. ``unpack`` and ``load`` unpacks from file-like object. :: >>> import msgpack >>> msgpack.packb([1, 2, 3]) '\x93\x01\x02\x03' >>> msgpack.unpackb(_) [1, 2, 3] ``unpack`` unpacks msgpack's array to Python's list, but can unpack to tuple:: >>> msgpack.unpackb(b'\x93\x01\x02\x03', use_list=False) (1, 2, 3) You should always pass the ``use_list`` keyword argument. See performance issues relating to use_list_ below. Read the docstring for other options. Streaming unpacking ^^^^^^^^^^^^^^^^^^^ ``Unpacker`` is a "streaming unpacker". It unpacks multiple objects from one stream (or from bytes provided through its ``feed`` method). :: import msgpack from io import BytesIO buf = BytesIO() for i in range(100): buf.write(msgpack.packb(range(i))) buf.seek(0) unpacker = msgpack.Unpacker(buf) for unpacked in unpacker: print unpacked Packing/unpacking of custom data type ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ It is also possible to pack/unpack custom data types. Here is an example for ``datetime.datetime``. :: import datetime import msgpack useful_dict = { "id": 1, "created": datetime.datetime.now(), } def decode_datetime(obj): if b'__datetime__' in obj: obj = datetime.datetime.strptime(obj["as_str"], "%Y%m%dT%H:%M:%S.%f") return obj def encode_datetime(obj): if isinstance(obj, datetime.datetime): return {'__datetime__': True, 'as_str': obj.strftime("%Y%m%dT%H:%M:%S.%f")} return obj packed_dict = msgpack.packb(useful_dict, default=encode_datetime) this_dict_again = msgpack.unpackb(packed_dict, object_hook=decode_datetime) ``Unpacker``'s ``object_hook`` callback receives a dict; the ``object_pairs_hook`` callback may instead be used to receive a list of key-value pairs. Extended types ^^^^^^^^^^^^^^^ It is also possible to pack/unpack custom data types using the msgpack 2.0 feature. >>> import msgpack >>> import array >>> def default(obj): ... if isinstance(obj, array.array) and obj.typecode == 'd': ... return msgpack.ExtType(42, obj.tostring()) ... raise TypeError("Unknown type: %r" % (obj,)) ... >>> def ext_hook(code, data): ... if code == 42: ... a = array.array('d') ... a.fromstring(data) ... return a ... return ExtType(code, data) ... >>> data = array.array('d', [1.2, 3.4]) >>> packed = msgpack.packb(data, default=default) >>> unpacked = msgpack.unpackb(packed, ext_hook=ext_hook) >>> data == unpacked True Advanced unpacking control ^^^^^^^^^^^^^^^^^^^^^^^^^^ As an alternative to iteration, ``Unpacker`` objects provide ``unpack``, ``skip``, ``read_array_header`` and ``read_map_header`` methods. The former two read an entire message from the stream, respectively deserialising and returning the result, or ignoring it. The latter two methods return the number of elements in the upcoming container, so that each element in an array, or key-value pair in a map, can be unpacked or skipped individually. Each of these methods may optionally write the packed data it reads to a callback function: :: from io import BytesIO def distribute(unpacker, get_worker): nelems = unpacker.read_map_header() for i in range(nelems): # Select a worker for the given key key = unpacker.unpack() worker = get_worker(key) # Send the value as a packed message to worker bytestream = BytesIO() unpacker.skip(bytestream.write) worker.send(bytestream.getvalue()) Note about performance ------------------------ GC ^^ CPython's GC starts when growing allocated object. This means unpacking may cause useless GC. You can use ``gc.disable()`` when unpacking large message. `use_list` option ^^^^^^^^^^^^^^^^^^ List is the default sequence type of Python. But tuple is lighter than list. You can use ``use_list=False`` while unpacking when performance is important. Python's dict can't use list as key and MessagePack allows array for key of mapping. ``use_list=False`` allows unpacking such message. Another way to unpacking such object is using ``object_pairs_hook``. Test ---- MessagePack uses `pytest` for testing. Run test with following command: $ py.test .. vim: filetype=rst
msgpack whl文件是一种用于安装`msgpack`Python安装包文件。whl文件是Python的一种标准分发格式,其全称为"wheel",它是Python包管理器的一种打包格式。 `msgpack`是一种序列化,它可以将Python的对象序列化为二进制数据,然后再将其反序列化回Python对象。这种序列化方式具有高效性和可读性,并且可以跨多种编程语言使用使用`msgpack`可以在不同的系统之间传递数据,并且可以实现快速的数据交换和存储。它支持多种数据类型,包括整数、浮点数、字符串、列表、字典等等。与其他序列化方式相比,`msgpack`的数据大小更小,并且速度更快。 要使用`msgpack`,我们需要将其安装到Python环境。而.whl文件则是一种用于Python包的安装文件格式。在安装whl文件之前,我们需要先确保已经安装了Python解释器和pip工具。 要使用whl文件安装msgpack,可以通过以下步骤进行: 1. 在Python环境打开终端或命令提示符。 2. 切换到存放.whl文件的目录。 3. 运行以下命令: `pip install msgpack.whl` 执行上述命令后,pip将会自动从.whl文件提取`msgpack`,并将其安装到Python环境。安装完成后,我们就可以在Python程序使用`msgpack`了。 总之,msgpack whl文件是一种用于安装msgpackPython安装包文件,它可以实现高效的数据序列化和反序列化,并且支持多种数据类型。使用whl文件可以方便地将msgpack安装到Python环境,从而在程序使用
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值