Python 编译阶段 -- 从 CST 到 AST

最新推荐文章于 2024-09-05 00:15:00 发布

zp0int

最新推荐文章于 2024-09-05 00:15:00 发布

阅读量988

点赞数

分类专栏： python internal 文章标签： cpython python c++

本文链接：https://blog.csdn.net/qq_31720329/article/details/107526652

版权

本文探讨了Python编译过程中的CST（抽象语法森林）到AST（抽象语法树）的转换，重点分析了相关源文件位置，如Python/ast.c和Python/pythonrun.c，并详细解释了CST到AST的转换函数在Python-ast.c中的实现。此外，还提供了转换前后结构的展示和用于查看AST结构的函数。

摘要由CSDN通过智能技术生成

CST TO AST

以后都在 github 更新，请戳 CST 到 AST

相关位置文件

Python/ast.c
Python/pythonrun.c
Include/Python-ast.h
Python/Python-ast.c
Python/asdl.c
Include/asdl.h

下面的命令会从 Parser/Python.asdl 中生成 Include/Python-ast.h 和 Python/Python-ast.c, 这两个自动生成的文件的结构体会被用来从之前的语法树中生成 AST

% make regen-ast
# Regenerate Include/Python-ast.h using Parser/asdl_c.py -h
./install-sh -c -d ./Include
python3 ./Parser/asdl_c.py \
                -h ./Include/Python-ast.h.new \
                ./Parser/Python.asdl
python3 ./Tools/scripts/update_file.py ./Include/Python-ast.h ./Include/Python-ast.h.new
# Regenerate Python/Python-ast.c using Parser/asdl_c.py -c
./install-sh -c -d ./Python
python3 ./Parser/asdl_c.py \
                -c ./Python/Python-ast.c.new \
                ./Parser/Python.asdl
python3 ./Tools/scripts/update_file.py ./Python/Python-ast.c ./Python/Python-ast.c.new

小写名称是 non-terminals.

大写名称是 terminals

文本 tokens 用双引号引起来

[] 表示 >= 0 个

{} 表示 >= 1 个

? 表示可能有可能没有, * 表示 >= 0 个

pythonrun

我们以交互式循环为例, 展开 pythonrun 的调用栈看看

pythonrun

CST 到 AST

我们把关注点放到 PyAST_FromNodeObject 上

如果我们执行

a = 2

这是上述语句生成的 CST 结构

n_type: 256, n_type_str: single_input, n_str: (null), n_children: 1
  n_type: 270, n_type_str: simple_stmt, n_str: (null), n_children: 2
    n_type: 271, n_type_str: small_stmt, n_str: (null), n_children: 1
      n_type: 272, n_type_str: expr_stmt, n_str: (null), n_children: 3
        n_type: 274, n_type_str: testlist_star_expr, n_str: (null), n_children: 1
          n_type: 305, n_type_str: test, n_str: (null), n_children: 1
            n_type: 309, n_type_str: or_test, n_str: (null), n_children: 1
              n_type: 310, n_type_str: and_test, n_str: (null), n_children: 1
                n_type: 311, n_type_str: not_test, n_str: (null), n_children: 1
                  n_type: 312, n_type_str: comparison, n_str: (null), n_children: 1
                    n_type: 315, n_type_str: expr, n_str: (null), n_children: 1
                      n_type: 316, n_type_str: xor_expr, n_str: (null), n_children: 1
                        n_type: 317, n_type_str: and_expr, n_str: (null), n_children: 1
                          n_type: 318, n_type_str: shift_expr, n_str: (null), n_children: 1
                            n_type: 319, n_type_str: arith_expr, n_str: (null), n_children: 1
                              n_type: 320, n_type_str: term, n_str: (null), n_children: 1
                                n_type: 321, n_type_str: factor, n_str: (null), n_children: 1
                                  n_type: 322, n_type_str: power, n_str: (null), n_children: 1
                                    n_type: 323, n_type_str: atom_expr, n_str: (null), n_children: 1