(Python编程)Pickle对象

Programming Python, 3rd Edition 翻译
最新版本见：http://wiki.woodpecker.org.cn/moin/PP3eD

 

   

19.4. Pickled Objects

19.4. Pickle对象

Probably the biggest limitation of DBM keyed files is in what they can store: data stored under a key must be a simple text string. If you want to store Python objects in a DBM file, you can sometimes manually convert them to and from strings on writes and reads (e.g., with str and eval calls), but this takes you only so far. For arbitrarily complex Python objects such as class instances and nested data structures, you need something more. Class instance objects, for example, cannot be later re-created from their standard string representations. Custom to-string conversions are error prone and not general.

DBM键控文件（DBM keyed file）最大的限制也许在于他们可以存储的东西：一个键值下存储的数据必须是个简单文本字符串。如果您想要在DBM文件中储存Python对象，有时您可以在读写的时候，手动进行与字符串的转换（例如，用str和eval调用），但只能做到这样。对任意复杂的Python对象，如类实例和嵌套的数据结构，您需要更多的东西。例如，类实例对象以后无法从其标准字符串表达（string representation）重建。自定义的到字符串的转换容易出错，并且不通用。

The Python pickle module, a standard part of the Python system, provides the conversion step needed. It converts nearly arbitrary Python in-memory objects to and from a single linear string format, suitable for storing in flat files, shipping across network sockets between trusted sources, and so on. This conversion from object to string is often called serializationarbitrary data structures in memory are mapped to a serial string form.

Python系统的标准部件，pickle模块，提供了所需的转换步骤。它可以将几乎任意的Python内存对象，转换为单一线性的字符串格式，使之适于无格式文件存储，或在可靠来源之间跨越网络套接口传输等等，并可反向转换。这种从对象到字符串的转换通常被称为序列化（serialization）：将内存中的任意数据结构映射为串行字符串形式。

The string representation used for objects is also sometimes referred to as a byte stream, due to its linear format. It retains all the content and references structure of the original in-memory object. When the object is later re-created from its byte string, it will be a new in-memory object identical in structure and value to the original, though located at a different memory address. The re-created object is effectively a copy of the original.

对象的字符串表达由于其线性的格式，有时也被称为字节流。它包含了原始内存中对象的所有内容和引用结构。当对象后来从其字节串重建时，内存中新建的对象与原对象具有相同的结构和值，但位于不同的内存地址。该重建对象实际上是原对象的复制。

Pickling works on almost any Python datatypenumbers, lists, dictionaries, class instances, nested structures, and moreand so is a general way to store data. Because pickles contain native Python objects, there is almost no database API to be found; the objects stored are processed with normal Python syntax when they are later retrieved.

Pickle可用于几乎所有的Python数据类型：数字、列表、字典、类实例、嵌套结构，等等，因此它是存储数据的通用方法。因为pickle包含的是Python本地对象，所以几乎没有数据库的API；对象存储与处理及后来的提取用的都是通常的Python语法。

19.4.1. Using Object Pickling

19.4.1. 使用对象pickle

Pickling may sound complicated the first time you encounter it, but the good news is that Python hides all the complexity of object-to-string conversion. In fact, the pickle module 's interfaces are incredibly simple to use. For example, to pickle an object into a serialized string, we can either make a pickler and call its methods or use convenience functions in the module to achieve the same effect:

第一次听到pickle，可能觉得有点复杂，但好消息是，Python隐藏了所有从对象到字符串转换的复杂性。事实上，pickle模块的接口简单易用，简直令人难以置信。例如，要pickle对象到一个序列化字符串，我们可以生成一个pickler，并调用其方法，或使用模块中的便捷函数来达到相同的效果：

P = pickle.Pickler( file)

Make a new pickler for pickling to an open output file object file.

生成一个新的pickler，用来pickle到一个打开的输出文件对象file。


P.dump( object)

Write an object onto the pickler's file/stream.

写一个对象到pickler的文件/流。


pickle.dump( object, file)

Same as the last two calls combined: pickle an object onto an open file.

等同于上两个调用的组合：pickle对象到一个打开的文件。


string = pickle.dumps( object)

Return the pickled representation of object as a character string.

返回一个字符串作为已pickle对象的表达。


Unpickling from a serialized string back to the original object is similarboth object and convenience function interfaces are available:

从一个序列化字符串unpickle回原始对象是类似的，可以用对象也可以用便捷函数接口：


U = pickle.Unpickler( file)

Make an unpickler for unpickling from an open input file object file.

生成一个unpickler，用来从一个打开的文件对象file unpickle。


object = U.load( )

Read an object from the unpickler's file/stream.

从unpickler的文件/流读取一个对象。


object = pickle.load( file)

Same as the last two calls combined: unpickle an object from an open file.

等同于上两个调用的组合：从一个打开的文件unpickle一个对象。


object = pickle.loads( string)

Read an object from a character string rather than a file.

从字符串读取一个对象，而不是从文件。


Pickler and Unpickler are exported classes. In all of the preceding cases, file is either an open file object or any object that implements the same attributes as file objects:

Pickler和Unpickler是导出类。在上述所有情况下，file是个已打开的文件对象，或者是实现了以下文件对象属性的任何对象：

Pickler calls the file's write method with a string argument.

Pickler会调用文件的write方法，参数是个字符串。

Unpickler calls the file's read method with a byte count, and readline without arguments.

Unpickler会调用文件的read方法，参数是字节数，以及readline，无参数。

Any object that provides these attributes can be passed in to the file parameters. In particular, file can be an instance of a Python class that provides the read/write methods (i.e., the expected file-like interface). This lets you map pickled streams to in-memory objects with classes, for arbitrary use. For instance, the StringIO standard library module discussed in Chapter 3 provides classes that map file calls to and from in-memory strings.

任何提供这些属性的对象都可以作为file参数传入。特别是，file可以是一个提供了读/写方法的Python类实例（即预期的类似文件的接口）。这让您可以用类映射pickle流到内存对象，并可任意使用。例如，第3章讨论的标准库模块StringIO提供的类，它们可映射文件调用到内存字符串或反之。

This hook also lets you ship Python objects across a network, by providing sockets wrapped to look like files in pickle calls at the sender, and unpickle calls at the receiver (see the sidebar "Making Sockets Look Like Files," in Chapter 13, for more details). In fact, for some, pickling Python objects across a trusted network serves as a simpler alternative to network transport protocols such as SOAP and XML-RPC; provided that Python is on both ends of the communication (pickled objects are represented with a Python-specific format, not with XML text).

该挂钩也可以让您通过网络传输Python对象，只要封装套接口，使之看上去像发送端pickle调用中的文件，以及像接收端unpickle调用中的文件（详见第13章侧栏“使套接口看上去像文件”）。事实上，对一些人来说，pickle Python对象并在一个值得信赖的网络上传输，是替代如SOAP和XML-RPC之类网络传输协议的一个简单方法；只要通信的两端都有Python（被pickle的对象是用Python专有的格式表达的，而不是用XML文本）。

19.4.2. Picking in Action

19.4.2. Pickle实战

In more typical use, to pickle an object to a flat file, we just open the file in write mode and call the dump function:

典型的使用情况是，pickle对象到无格式文件，我们只需以写模式打开文件，并调用dump函数：

% python
>>> table = {'a': [1, 2, 3],
             'b': ['spam', 'eggs'],
             'c': {'name':'bob'}}
>>>
>>> import pickle
>>> mydb  = open('dbase', 'w')
>>> pickle.dump(table, mydb)



Notice the nesting in the object pickled herethe pickler handles arbitrary structures. To unpickle later in another session or program run, simply reopen the file and call load:

注意这个被pickle对象中的嵌套：pickler可以处理任意结构。然后，在另一个会话或程序中unpickle，只要重新打开该文件，并调用load：

% python
>>> import pickle
>>> mydb  = open('dbase', 'r')
>>> table = pickle.load(mydb)
>>> table
{'b': ['spam', 'eggs'], 'a': [1, 2, 3], 'c': {'name': 'bob'}}



The object you get back from unpickling has the same value and reference structure as the original, but it is located at a different address in memory. This is true whether the object is unpickled in the same or a future process. In Python-speak, the unpickled object is == but is not is:

unpickle所得的对象具有与原对象相同的值和引用结构，但它位于不同的内存地址。无论在同一进程或另一进程unpickle，都是这样。用Python的话来说，unpickle后的对象是“==”关系，但不是“is”关系：

% python
>>> import pickle
>>> f = open('temp', 'w')
>>> x = ['Hello', ('pickle', 'world')]           # list with nested tuple
>>> pickle.dump(x, f)
>>> f.close( )                                   # close to flush changes
>>>
>>> f = open('temp', 'r')
>>> y = pickle.load(f)
>>> y
['Hello', ('pickle', 'world')]
>>>
>>> x == y, x is y
(True, False)



To make this process simpler still, the module in Example 19-1 wraps pickling and unpickling calls in functions that also open the files where the serialized form of the object is stored.

为了让这一过程更简单，例19-1中的模块把pickle和unpickle调用封装在函数中，在函数中同时还打开文件，并将对象的序列化存储在该文件中。

Example 19-1. PP3E/Dbase/filepickle.py
import pickle

def saveDbase(filename, object):
    file = open(filename, 'w')
    pickle.dump(object, file)          # pickle to file
    file.close( )                     # any file-like object will do

def loadDbase(filename):
    file = open(filename, 'r')
    object = pickle.load(file)         # unpickle from file
    file.close( )                     # re-creates object in memory
    return object


 


To store and fetch now, simply call these module functions; here they are in action managing a fairly complex structure with multiple references to the same nested objectthe nested list called L at first is stored only once in the file:

现在，存储和提取时只需调用这些模块函数；以下实例是个相当复杂的结构，具有对同一嵌套对象的多重引用，该嵌套列表，即第1个L，在文件中只会保存一次：

C:/.../PP3E/Dbase>python
>>> from filepickle import *
>>> L = [0]
>>> D = {'x':0, 'y':L}
>>> table = {'A':L, 'B':D}              # L appears twice
>>> saveDbase('myfile', table)          # serialize to file

C:/.../PP3E/Dbase>python
>>> from filepickle import *
>>> table = loadDbase('myfile')         # reload/unpickle
>>> table
{'B': {'x': 0, 'y': [0]}, 'A': [0]}
>>> table['A'][0] = 1                   # change shared object
>>> saveDbase('myfile', table)          # rewrite to the file

C:/.../PP3E/Dbase>python
>>> from filepickle import *
>>> print loadDbase('myfile')           # both L's updated as expected
{'B': {'x': 0, 'y': [1]}, 'A': [1]}



Besides built-in types like the lists, tuples, and dictionaries of the examples so far, class instances may also be pickled to file-like objects. This provides a natural way to associate behavior with stored data (class methods process instance attributes) and provides a simple migration path (class changes made in module files are automatically picked up by stored instances). Here's a brief interactive demonstration:

除了内置的类型，如以上例子中的列表，元组和字典，类实例也可被pickle到类似文件的对象中。这提供了一个自然的方式来关联行为与存储的数据（类方法处理实例的属性），并提供了一条简单的迁移路径（被存储的实例将自动获得模块文件中对类的更改）。以下是个简短的交互演示：

>>> class Rec:
        def _ _init_ _(self, hours):
            self.hours = hours
        def pay(self, rate=50):
            return self.hours * rate

>>> bob = Rec(40)
>>> import pickle
>>> pickle.dump(bob, open('bobrec', 'w'))
>>>
>>> rec = pickle.load(open('bobrec'))
>>> rec.hours
40
>>> rec.pay( )
2000



We'll explore how this works in more detail in conjunction with shelves later in this chapteras we'll see, although the pickle module can be used directly, it is also the underlying translation engine in both shelves and ZODB databases.

我们将与本章下面的shelve一起详细探讨这是如此工作的。我们将会看到，虽然pickle模块可直接使用，但它也是shelve和ZODB数据库的底层翻译引擎。

In fact, Python can pickle just about anything, except for:

事实上，Python可以pickle任何东西，除了：

Compiled code objects; functions and classes record just their names in pickles, to allow for later reimport and automatic acquisition of changes made in module files.

编译的代码对象；函数和类在pickle中只是记录了它们的名字，以便后来重新导入和自动获取模块文件中的更改。

Instances of classes that do not follow class importability rules (more on this at the end of the section "Shelve Files," later in this chapter).

不遵守类可导入规则（class importability rule）的类实例（本章后面“Shelve文件”一节的尾部有更多相关内容）。

Instances of some built-in and user-defined types that are coded in C or depend upon transient operating system states (e.g., open file objects cannot be pickled).

用C编码的或依赖于操作系统瞬态的一些内置的和用户定义类型的实例（例如，打开的文件对象无法pickle）。

A PicklingError is raised if an object cannot be pickled.

如果对象不能pickle，会引发PickleError。

19.4.3. Pickler Protocols and cPickle

19.4.3. Pickler协议和cPickle

In recent Python releases, the pickler introduced the notion of protocolsstorage formats for pickled data. Specify the desired protocol by passing an extra parameter to the pickling calls (but not to unpickling calls: the protocol is automatically determined from the pickled data):

在最近的Python版本中，pickler推出了协议的概念：pickle数据的保存格式。通过pickle调用时传入一个额外的参数，可指定所需的协议（但unpickle调用不需要：协议是自动从已pickle的数据确定的）：

pickle.dump(object, file, protocol)



Pickled data may be created in either text or binary protocols. By default, the storage protocol is text (also known as protocol 0). In text mode, the files used to store pickled objects may be opened in text mode as in the earlier examples, and the pickled data is printable ASCII text, which can be read (it's essentially instructions for a stack machine).

Pickle数据可以按文本协议或二进制协议产生。默认情况下，存储协议是文本协议（也称为0号协议）。在文本模式下，用来存储pickle对象的文件可以用文本模式打开，如上述的例子，并且pickle的数据是可打印的ASCII文本，并且是可读的（这基本上是对堆栈机实现的指示）。

The alternative protocols (protocols 1 and 2) store the pickled data in binary format and require that files be opened in binary mode (e.g., rb, wb). Protocol 1 is the original binary format; protocol 2, added in Python 2.3, has improved support for pickling of new-style classes. Binary format is slightly more efficient, but it cannot be inspected. An older option to pickling calls, the bin argument, has been subsumed by using a pickling protocol higher than 0. The pickle module also provides a HIGHEST_PROTOCOL variable that can be passed in to automatically select the maximum value.

其他协议（1号和2号协议 ）以二进制格式存储pickle数据，并要求文件以二进制模式打开（例如：rb、wb）。1号协议是原始二进制格式；2号协议是Python 2.3增加的，它改善了对新型类pickle的支持。二进制格式效率更高一点，但它无法进行查看。旧的pickle调用有一个选项，即bin参数，现已被归入使用大于0的协议。pickle模块还提供了一个HIGHEST_PROTOCOL变量，传入它可以自动选择最大的协议值。

One note: if you use the default text protocol, make sure you open pickle files in text mode later. On some platforms, opening text data in binary mode may cause unpickling errors due to line-end formats on Windows:

注意：如果您使用默认的文本协议，以后请务必以文本模式打开pickle文件。在一些平台上，因为Windows的行尾格式不同，以二进制模式打开文本数据可能会导致unpickle错误:

>>> f = open('temp', 'w')                  # text mode file on Windows
>>> pickle.dump(('ex', 'parrot'), f)       # use default text protocol
>>> f.close( )
>>>
>>> pickle.load(open('temp', 'r'))         # OK in text mode
('ex', 'parrot')
>>> pickle.load(open('temp', 'rb'))        # fails in binary
Traceback (most recent call last):
  File "<pyshell#337>", line 1, in -toplevel-
    pickle.load(open('temp', 'rb'))
 ...lines deleted...
ValueError: insecure string pickle



One way to sidestep this potential issue is to always use binary mode for your files, even for the text pickle protocol. Since you must open files in binary mode for the binary pickler protocols anyhow (higher than the default 0), this isn't a bad habit to get into:

回避这个潜在问题的方法之一是，总是使用二进制模式的文件，即使是用文本pickle协议。至少对于二进制pickler协议（高于默认0），您必须以二进制模式打开文件，所以这不是一个坏习惯：

>>> f = open('temp', 'wb')                 # create in binary mode
>>> pickle.dump(('ex', 'parrot'), f)       # use text protocol
>>> f.close( )
>>>
>>> pickle.load(open('temp', 'rb'))
('ex', 'parrot')
>>> pickle.load(open('temp', 'r'))
('ex', 'parrot')



Refer to Python's library manual for more information on the pickler. Also check out marshal, a module that serializes an object too, but can handle only simple object types. pickle is more general than marshal and is normally preferred.

请参考Python库手册，以了解更多pickler的信息。另外，请查阅marshal，它也是一个序列化对象的模块，但只能处理简单对象类型。pickle比marshal更通用，并通常是首选。

And while you are flipping (or clicking) through that manual, be sure to also see the entries for the cPickle modulea reimplementation of pickle coded in C for faster performance. You can explicitly import cPickle for a substantial speed boost; its chief limitation is that you cannot subclass its versions of Pickle and Unpickle because they are functions, not classes (this is not required by most programs). The pickle and cPickle modules use compatible data formats, so they may be used interchangeably.

而当你翻看（或点击）Python手册时，请一定也要看看cPickle模块的条目，它是pickle的C语言实现，性能上更快。您可以显式导入cPickle替代pickle，以大幅提升速度；其主要的限制是，你不能继承该版本的Pickle和Unpickle，因为它们是函数，而不是类（多数程序并不要求它们是类）。pickle和cPickle模块使用兼容的数据格式，所以它们可以互换使用。

If it is available in your Python, the shelve module automatically chooses the cPickle module for faster serialization, instead of pickle. I haven't explained shelve yet, but I will now.
 
如果您的Python中有shelve模块，它会自动选用cPickle模块，而不是pickle，以达到更快的序列化。我还没有解释过shelve，但我马上就会讲到它。