objgraph - Python Object Graphs - 内存泄漏 (Memory Leak)

objgraph - Python Object Graphs - 内存泄漏 (Memory Leak)

https://pypi.org/project/objgraph/
https://github.com/mgedmin/objgraph

objgraph is a module that lets you visually explore Python object graphs.
objgraph 是一个模块，可让您直观地浏览 Python 对象图。

1. Installation

sudo pip3 install objgraph

strong@foreverstrong:~$ sudo pip3 install objgraph
[sudo] password for strong: 
The directory '/home/strong/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/strong/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting objgraph
  Downloading https://files.pythonhosted.org/packages/7d/21/b8ea10bea21a3ecb603ab0a8a59e49282d83eadba16e47464193b0b70dce/objgraph-3.4.1-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): graphviz in /usr/local/lib/python3.5/dist-packages (from objgraph)
Installing collected packages: objgraph
Successfully installed objgraph-3.4.1
strong@foreverstrong:~$

2. Documentation

https://mg.pov.lt/objgraph/

2.1 Quick start

Try this in a Python shell:

strong@foreverstrong:~$ python3
Python 3.5.2 (default, Oct  8 2019, 13:06:37) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> x = []
>>> x
[]
>>> y = [x, [x], dict(x=x)]
>>> y
[[], [[]], {'x': []}]
>>> 
>>> import objgraph
>>> objgraph.show_refs([y], filename='sample-graph.png')
Graph written to /tmp/objgraph-txh1vhmd.dot (4 nodes)
Image generated as sample-graph.png
>>>

If you’ve installed xdot, omit the filename argument to get the interactive viewer.
如果您已经安装了 xdot，请省略 filename 参数以获取交互式查看器。

You should see a graph like this:

If you prefer to handle your own file output, you can provide a file object to the output parameter of show_refs and show_backrefs instead of a filename. The contents of this file will contain the graph source in DOT format.
如果您喜欢处理自己的文件输出，则可以向 show_refs and show_backrefs 的 output 参数提供文件对象，而不是文件名。该文件的内容将包含 DOT 格式的图形源。

2.2 Backreferences

strong@foreverstrong:~$ python3
Python 3.5.2 (default, Oct  8 2019, 13:06:37) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> x = []
>>> x
[]
>>> y = [x, [x], dict(x=x)]
>>> y
[[], [[]], {'x': []}]
>>> 
>>> import objgraph
>>> objgraph.show_refs([y], filename='sample-graph.png')
Graph written to /tmp/objgraph-txh1vhmd.dot (4 nodes)
Image generated as sample-graph.png
>>> 
>>> objgraph.show_backrefs([x], filename='sample-backref-graph.png')
Graph written to /tmp/objgraph-5irdq6jp.dot (18 nodes)
Image generated as sample-backref-graph.png
>>>

and you’ll see

2.3 Memory leak example (内存泄漏)

The original purpose of objgraph was to help me find memory leaks. The idea was to pick an object in memory that shouldn’t be there and then see what references are keeping it alive.
objgraph 的最初目的是帮助我发现内存泄漏。这个想法是要在内存中选择一个不应该存在的对象，然后查看哪些引用使该对象保持活动状态。

To get a quick overview of the objects in memory, use the imaginatively-named show_most_common_types():
要快速了解内存中的对象，请使用富有想象力的名称 show_most_common_types()：

>>> objgraph.show_most_common_types() # doctest: +RANDOM_OUTPUT
tuple                      5224
function                   1329
wrapper_descriptor         967
dict                       790
builtin_function_or_method 658
method_descriptor          340
weakref                    322
list                       168
member_descriptor          167
type                       163

strong@foreverstrong:~$ python3
Python 3.5.2 (default, Oct  8 2019, 13:06:37) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> x = []
>>> x
[]
>>> y = [x, [x], dict(x=x)]
>>> y
[[], [[]], {'x': []}]
>>> 
>>> import objgraph
>>> objgraph.show_refs([y], filename='sample-graph.png')
Graph written to /tmp/objgraph-txh1vhmd.dot (4 nodes)
Image generated as sample-graph.png
>>> 
>>> objgraph.show_backrefs([x], filename='sample-backref-graph.png')
Graph written to /tmp/objgraph-5irdq6jp.dot (18 nodes)
Image generated as sample-backref-graph.png
>>> 
>>> objgraph.show_most_common_types()
function                   2149
dict                       1130
wrapper_descriptor         985
tuple                      924
weakref                    828
builtin_function_or_method 706
method_descriptor          691
set                        386
getset_descriptor          363
list                       331
>>> 
>>> exit()
strong@foreverstrong:~$

But that’s looking for a small needle in a large haystack. Can we limit our haystack to objects that were created recently? Perhaps.
但这就是在大海捞针中寻找一根小针。我们可以将干草堆限制为最近创建的对象吗？也许。

needle [ˈniːdl]：n. 针，指针，刺激，针状物 vi. 缝纫，做针线 vt. 刺激，用针缝
haystack [ˈheɪstæk]：n. 干草堆，比喻如大海捞针般难找

Let’s define a function that “leaks” memory
让我们定义一个泄漏内存的函数

>>> class MyBigFatObject(object):
...     pass
...
>>> def computate_something(_cache={}):
...     _cache[42] = dict(foo=MyBigFatObject(),
...                       bar=MyBigFatObject())
...     # a very explicit and easy-to-find "leak" but oh well
...     x = MyBigFatObject() # this one doesn't leak

We take a snapshot of all the objects counts that are alive before we call our function
在调用函数之前，我们会对所有活动的对象计数进行快照

>>> objgraph.show_growth(limit=3) # doctest: +RANDOM_OUTPUT
tuple                  5228     +5228
function               1330     +1330
wrapper_descriptor      967      +967

and see what changes after we call it

>>> computate_something()
>>> objgraph.show_growth() # doctest: +RANDOM_OUTPUT
MyBigFatObject        2        +2
dict                797        +1

It’s easy to see MyBigFatObject instances that appeared and were not freed. I can pick one of them at random and trace the reference chain back to one of the garbage collector’s roots.
很容易看到 MyBigFatObject 实例出现了但没有释放。我可以随机选择其中之一，并将引用链追溯到垃圾收集器的根之一。

freed：free 的过去式和过去分词
sake [seɪk; ˈsɑːki]：n. 目的，利益，理由，日本米酒

For simplicity’s sake let’s assume all of the roots are modules. objgraph provides a function, is_proper_module(), to check this. If you’ve any examples where that isn’t true, I’d love to hear about them (although see Reference counting bugs).
为了简单起见，让我们假设所有的根都是模块。objgraph 提供了一个函数 is_proper_module() 进行检查。如果您有任何不正确的示例，我很想听听它们 （尽管请参阅 Reference counting bugs）。

>>> import random
>>> objgraph.show_chain(
...     objgraph.find_backref_chain(
...         random.choice(objgraph.by_type('MyBigFatObject')),
...         objgraph.is_proper_module),
...     filename='chain.png')
Graph written to ...dot (13 nodes)
Image generated as chain.png

It is perhaps surprising to find linecache at the end of that chain (apparently doctest monkey-patches it), but the important things - computate_something() and its cache dictionary - are in there.
在该链的末尾找到线缓存可能很令人惊讶 (apparently doctest monkey-patches it)，但是重要的东西 - computate_something() 及其缓存字典就在其中。

There are other tools, perhaps better suited for memory leak hunting: heapy, Dozer.
还有其他工具，也许更适合于内存泄漏搜寻：heapy, Dozer。

2.4 Reference counting bugs

Bugs in C-level reference counting may leave objects in memory that do not have any other objects pointing at them. You can find these by calling get_leaking_objects(), but you’ll have to filter out legitimate GC roots from them, and there are a lot of those:
C 级引用计数中的错误可能会将没有任何其他对象指向它们的对象保留在内存中。您可以通过调用 get_leaking_objects() 来找到它们，但是您必须从它们中过滤出合法的GC根，并且其中有很多：

>>> roots = objgraph.get_leaking_objects()
>>> len(roots) # doctest: +RANDOM_OUTPUT
4621

>>> objgraph.show_most_common_types(objects=roots)
... # doctest: +RANDOM_OUTPUT
tuple          4333
dict           171
list           74
instancemethod 4
listiterator   2
MemoryError    1
Sub            1
RuntimeError   1
Param          1
Add            1

>>> objgraph.show_refs(roots[:3], refcounts=True, filename='roots.png')
... # doctest: +NODES_VARY
Graph written to ...dot (19 nodes)
Image generated as roots.png