in python, use range
for loop is much faster than while
, why?
>>> timeit.timeit("for i in range(100): pass")
1.6118687279995356
>>> timeit.timeit("i = 0\nwhile i < 100: i += 1")
5.769564033998904
Let’s check the dis
output of range
and while
:
>>> dis.dis(compile("for i in range(100): pass", '<string>', 'exec'))
1 0 SETUP_LOOP 20 (to 23)
3 LOAD_NAME 0 (range)
6 LOAD_CONST 0 (100)
9 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
12 GET_ITER
>> 13 FOR_ITER 6 (to 22)
16 STORE_NAME 1 (i)
19 JUMP_ABSOLUTE 13
>> 22 POP_BLOCK
>> 23 LOAD_CONST 1 (None)
26 RETURN_VALUE
>>> dis.dis(compile("i = 0\nwhile i < 100: i += 1", '', 'exec'))
1 0 LOAD_CONST 0 (0)
3 STORE_NAME 0 (i)
2 6 SETUP_LOOP 26 (to 35)
>> 9 LOAD_NAME 0 (i)
12 LOAD_CONST 1 (100)
15 COMPARE_OP 0 (<)
18 POP_JUMP_IF_FALSE 34
21 LOAD_NAME 0 (i)
24 LOAD_CONST 2 (1)
27 INPLACE_ADD
28 STORE_NAME 0 (i)
31 JUMP_ABSOLUTE 9
>> 34 POP_BLOCK
>> 35 LOAD_CONST 3 (None)
38 RETURN_VALUE
>>>
the assembly code of range
is less than while
.
What do those codes do?
the return of compile
function is a code object
.
compile(source, filename, mode, flags=0, dont_inherit=False, optimize=-1)
filename
should give the file from which the code was read; pass some recognizable value if it wasn’t read from a file (‘’ is commonly used).
mode
:
– exec
: if source is a sequence of statements.
–eval
: if source is a single expression.
–single
: if source is a single interactive statement.
Difference between statement
and expression
?
expression
: always yields a value
statement
: makes up a line
also, expressions
are always statement
.
>>> co = compile("for i in range(100): pass", '<string>', 'exec')
>>> dir(co)
['__class__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'co_argcount', 'co_cellvars', 'co_code', 'co_consts', 'co_filename', 'co_firstlineno', 'co_flags', 'co_freevars', 'co_kwonlyargcount', 'co_lnotab', 'co_name', 'co_names', 'co_nlocals', 'co_stacksize', 'co_varnames']
refer to https://docs.python.org/3/library/inspect.html for the details of each attribute.
co_code
: bytecode byte string
>>> co.co_code
b'x\x14\x00e\x00\x00d\x00\x00\x83\x01\x00D]\x06\x00Z\x01\x00q\r\x00Wd\x01\x00S'
co_consts
: the constant table, includes all constants used in instruction.
>>> co.co_consts
(100, None)
co_names
: global variables
>>> co.co_names
('range', 'i')
co_nlocals
: number of local variables
>>> co.co_nlocals
0
co_varnames
: list of local variables
>>> co.co_varnames
()
co_argcount
: if code object
is of a function, this is the count of args.
>>> co.co_argcount
0
co_name
: function name
>>> co.co_name
'<module>'
co_cellvars
: cells attached, used when the code object
is from a closure
.
>>> co.co_cellvars
()
Explanation of range
:
1 | 2 | 3 | 4 | 5 | 6
---------------------------------------------------------------
1 0 SETUP_LOOP 20 (to 23)
3 LOAD_NAME 0 (range)
6 LOAD_CONST 0 (100)
9 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
12 GET_ITER
>> 13 FOR_ITER 6 (to 22)
16 STORE_NAME 1 (i)
19 JUMP_ABSOLUTE 13
>> 22 POP_BLOCK
>> 23 LOAD_CONST 1 (None)
26 RETURN_VALUE
>>>
1: line number in the source code
2: >>
means a JUMP
.
3: byte code counter
.
4: opname
, instruction name.
5: arguments to instruction from 4, which could be index to constant table: co_consts
, global vars: co_names
or used as code block expansion.
6: the human-friendly interpretation of the instruction argument
line1: means the block expands from byte code counter 0 to 23
SETUP_LOOP(delta)
Pushes a block for a loop onto the block stack. The block spans from the current instruction with a size of delta bytes.
line2: push range
to stack
LOAD_NAME(namei)
Pushes the value associated with co_names[namei] onto the stack.
line3: push 100
to stack
LOAD_CONST(consti)
Pushes co_consts[consti] onto the stack
line4: call function, which will pop out all args and function object and push return value.
CALL_FUNCTION(argc)
Calls a function. The low byte of argc indicates the number of positional parameters, the high byte the number of keyword parameters. On the stack, the opcode finds the keyword parameters first. For each keyword argument, the value is on top of the key. Below the keyword parameters, the positional parameters are on the stack, with the right-most parameter on top. Below the parameters, the function object to call is on the stack. Pops all function arguments, and the function itself off the stack, and pushes the return value.
line5: get iterator from iterable: TOS
, current top on stack is the range object.
GET_ITER
Implements TOS = iter(TOS).
line6: iterate through the iterator, jump to position in byte code counter 22 if iterator is exhausted – which means i
reaches 100.
FOR_ITER(delta)
TOS is an iterator. Call its __next__() method. If this yields a new value, push it on the stack (leaving the iterator below it). If the iterator indicates it is exhausted TOS is popped, and the byte code counter is incremented by delta.
line7: assign TOS to i
.
in the loop, TOS
is the return value of last __next__().
STORE_NAME(namei)
Implements name = TOS. namei is the index of name in the attribute co_names of the code object. The compiler tries to use STORE_FAST or STORE_GLOBAL if possible.
line8: jump to 13: FOR_ITER
.
JUMP_ABSOLUTE(target)
Set bytecode counter to target.
line9: remove the loop-used frame from stack.
POP_BLOCK
Removes one block from the block stack. Per frame, there is a stack of blocks, denoting nested loops, try statements, and such.
line10: push None
to stack, which will be the return value.
line11: pop off the return value, which is on the top of stack.
RETURN_VALUE
Returns with TOS to the caller of the function.
video to further introduce cpython
:
http://pyvideo.org/pycon-us-2012/stepping-through-cpython.html
Reference:
https://docs.python.org/3.5/library/dis.html#opcode-FOR_ITER
http://www.goldsborough.me/python/low-level/2016/10/04/00-31-30-disassembling_python_bytecode/
https://stackoverflow.com/questions/12673074/how-should-i-understand-the-output-of-dis-dis
https://late.am/post/2012/03/26/exploring-python-code-objects.html