python struct 学习笔记

python struct 学习笔记

part 1: introduce

python 是一门简洁的语言。为了与其他语言或平台（尤其在网络传输过程中）进行互相转换，有必要了解一下 python struct.

part 2: get help

NAME
struct
FILE
/usr/lib/python2.6/struct.py
FUNCTIONS
calcsize(...)
Return size of C struct described by format string fmt.
pack(...)
Return string containing values v1, v2, ... packed according to fmt.
pack_into(...)
Pack the values v1, v2, ... according to fmt.
Write the packed bytes into the writable buffer buf starting at offset.
unpack(...)
Unpack the string containing packed C structure data, according to fmt.
Requires len(string) == calcsize(fmt).
unpack_from(...)
Unpack the buffer, containing packed C structure data, according to
fmt, starting at offset. Requires len(buffer[offset:]) >= calcsize(fmt).


part 3: instances

function 1:
calcsize(...) 计算格式字符串对应的结果的长度

>>> b = struct.calcsize('HB')
>>> b
3
>>> b = struct.calcsize('!HBBBBBBIIIIBH')
>>> b
27
>>> b = struct.calcsize('HBBBBBBIIIIBH')
>>> b
28

function 2:
pack(...) 根据格式字符串，将python值转换为字节流（字节数组）

>>> a = struct.pack("!HBBBBBBIIIIBH",48251,0,24,0,1,0,0,842003,4294963512,2300851747,368142189,1,2)
>>> a
'\xbc{\x00\x18\x00\x01\x00\x00\x00\x0c\xd9\x13\xff\xff\xf18\x89$6#\x15\xf1gm\x01\x00\>>> repr(a)x02'

function 3:
pack_into(...) 与pack()类似。根据格式字符串，将python值转换为字节流（字节数组),并且将这字节流保存到一个buffer 中

>>> from ctypes import create_string_buffer
>>> buf = create_string_buffer(40)
>>> a = struct.pack_into("!HBBBBBBIIIIBH",buf,0,48251,0,24,0,1,0,0,842003,4294963512,2300851747,368142189,1,2)
>>> buf
<ctypes.c_char_Array_40 object at 0x7f0eefd11dd0>

function 4：
unpack(...) 与pack() 相反。根据格式字符串，将字节流转换成python 数据类型。

>>> b = struct.unpack("!HBBBBBBIIIIBH",a)

>>> b
(48251, 0, 24, 0, 1, 0, 0, 842003, 4294963512, 2300851747, 368142189, 1, 2)


function 5:
unpack_from(...) 与unpack()类似。根据格式字符串，从一个buffer 中将字节流转换成python 数据类型

>>> b = struct.unpack_from("!HBBBBBBIIIIBH",buf,0)
>>> b
(48251, 0, 24, 0, 1, 0, 0, 842003, 4294963512, 2300851747, 368142189, 1, 2)
>>> buf

<ctypes.c_char_Array_40 object at 0x7f0eefd11dd0>

# pack 之后得到的字节流（数组）也可以像序列一样来操作：

>>> a = struct.pack("!HBBBBBBIIIIBH",48251,0,24,0,1,0,0,842003,4294963512,2300851747,368142189,1,2)
>>> a
'\xbc{\x00\x18\x00\x01\x00\x00\x00\x0c\xd9\x13\xff\xff\xf18\x89$6#\x15\xf1gm\x01\x00\x02'
>>> len(a)
27
>>> head = a[:25]
>>> head
'\xbc{\x00\x18\x00\x01\x00\x00\x00\x0c\xd9\x13\xff\xff\xf18\x89$6#\x15\xf1gm\x01'
>>> command = a[25:]
>>> command
'\x00\x02'
>>> command = a[24:]
>>> command
'\x01\x00\x02'
>>> command = a[-4:]
>>> command
'm\x01\x00\x02'
>>> command = a[-3:]
>>> command
'\x01\x00\x02'

part 4: big-endian & format

Functions to convert between Python values and C structs represented
as Python strings. It uses format strings (explained below) as compact
descriptions of the lay-out of the C structs and the intended conversion
to/from Python values.

The optional first format char indicates byte order, size and alignment:
@: native order, size & alignment (default)
=: native order, std. size & alignment
<: little-endian, std. size & alignment
>: big-endian, std. size & alignment
!: same as >

The remaining chars indicate types of args and must match exactly;
these can be preceded by a decimal repeat count:
x: pad byte (no data); c:char; b:signed byte; B:unsigned byte;
?: _Bool (requires C99; if not available, char is used instead)
h:short; H:unsigned short; i:int; I:unsigned int;
l:long; L:unsigned long; f:float; d:double.
Special cases (preceding decimal count indicates length):
s:string (array of char); p: pascal string (with count byte).
Special case (only available in native format):
P:an integer type that is wide enough to hold a pointer.
Special case (not in native mode unless 'long long' in platform C):
q:long long; Q:unsigned long long
Whitespace between formats is ignored.

大端模式：

字节的传输约定：按照B7~B0的顺序传输；
字的传输约定： 先传递高8位（B15~B8），再传递低8位，（B7~B0）；
双字的传输约定： 先传递高24位，（B31~B24），然后传递高16位（B23~B16），再传递高8位（B15~B8），最后传递低8位（B7~B0）。

>>> b = struct.pack("!H",1)
>>> b
'\x00\x01'
>>> b = struct.pack("H",1)
>>> b
'\x01\x00'
>>> b = struct.pack("L",1)
>>> b
'\x01\x00\x00\x00\x00\x00\x00\x00'
>>> b = struct.pack("!L",1)
>>> b
'\x00\x00\x00\x01'


格式化字符串：

# P void * long
>>> buffer = struct.pack("P",1)
>>> buffer
'\x01\x00\x00\x00\x00\x00\x00\x00'

# p char[] string
>>> buffer = struct.pack("p","string")
>>> buffer
'\x00'

# s char[] string
>>> buffer = struct.pack("ssss","string")
>>> buffer
's'
>>> buffer = struct.pack("sss","string","st","ff")
>>> buffer
'ssf'
>>> buffer = struct.pack("sss","string","j","ff")
>>> buffer
'sjf'
>>> buffer = struct.pack("%ds"%len('jia xiaolei'),"jia xiaolei")
>>> buffer
'jia xiaolei'

# d double float
>>> buffer = struct.pack("d", 1.2)
>>> buffer
'333333\xf3?'

# f float float
>>> buffer = struct.pack("f", 1.2)
>>> buffer
'\x9a\x99\x99?'

# 这里很有用，Q 可以匹配几乎所有的东西
Q unsigned long long long
>>> buffer = struct.pack("Q",25111)
>>> buffer
'\x17b\x00\x00\x00\x00\x00\x00'
>>> buffer = struct.pack("Q",47882545111)
>>> buffer
'\xd7\xa7\x05&\x0b\x00\x00\x00'

# q long long long
>>> buffer = struct.pack("q",47882545111)
>>> buffer
'\xd7\xa7\x05&\x0b\x00\x00\x00'
>>> buffer = struct.pack("q",2545111)
>>> buffer
'\xd7\xd5&\x00\x00\x00\x00\x00'

# L unsigned long long

>>> buffer = struct.pack("L",2545111)
>>> buffer
'\xd7\xd5&\x00\x00\x00\x00\x00'
>>> buffer = struct.pack("L",47882545111)
>>> buffer
'\xd7\xa7\x05&\x0b\x00\x00\x00'

# l long integer
>>> buffer = struct.pack("l",47882545111)
>>> buffer
'\xd7\xa7\x05&\x0b\x00\x00\x00'
>>> buffer = struct.pack("l",2545111)
>>> buffer
'\xd7\xd5&\x00\x00\x00\x00\x00'
>>> buffer = struct.pack("l",47882545111)
>>> buffer
'\xd7\xa7\x05&\x0b\x00\x00\x00'
>>> buffer = struct.pack("l",2545111)
>>> buffer
'\xd7\xd5&\x00\x00\x00\x00\x00'

# I unsigned int integer or long

>>> buffer = struct.pack("I",2545111)
>>> buffer
'\xd7\xd5&\x00'
>>> buffer = struct.pack("I",47882545111)
__main__:1: DeprecationWarning: 'I' format requires 0 <= number <= 4294967295
# 如果传入的值的范围查过格式字符串说约定的范围，会报错

# i int integer
>>> buffer = struct.pack("i",5111)
>>> buffer
'\xf7\x13\x00\x00'
>>> buffer = struct.pack("i",775111)
>>> buffer
'\xc7\xd3\x0b\x00'

# H unsigned short integer
>>> buffer = struct.pack("H",775111)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
struct.error: ushort format requires 0 <= number <= USHRT_MAX
>>> buffer = struct.pack("H",11)
>>> buffer
'\x0b\x00'
>>> buffer = struct.pack("H",711)
>>> buffer
'\xc7\x02'

# h short integer
>>> buffer = struct.pack("h",711)
>>> buffer
'\xc7\x02'
>>> buffer = struct.pack("",7811)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
struct.error: pack requires exactly 0 arguments

# ? _Bool bool
>>> buffer = struct.pack("?",True)
>>> buffer
'\x01'
>>> buffer = struct.pack("?",False)
>>> buffer
'\x00'

# B unsigned char integer
>>> buffer = struct.pack("B",2)
>>> buffer
'\x02'
>>> buffer = struct.pack("B",15)
>>> buffer
'\x0f'

# b signed char integer
>>> buffer = struct.pack("b",15)
>>> buffer
'\x0f'
>>> buffer = struct.pack("b",1)
>>> buffer
'\x01'
>>> buffer = struct.pack("b",51)
>>> buffer
'3'
>>> buffer = struct.pack("b",51)
>>> buffer
'3'

# c char string of length 1
>>> buffer = struct.pack("c","1")
>>> buffer
'1'
>>> buffer = struct.pack("c","7")
>>> buffer
'7'

# x
>>> buffer = struct.pack("x",)
>>> buffer
'\x00'