译者按:从这一章开始,我们要准备随时开车了。所以有些不便于翻译的专有名词。我就保留了。
迄今为止,我们弄明白了:
- 把句子分解成token。这个过程也叫词法分析,干这个事儿的模块叫做词法分析器。我们也从头写过了一个自己的词法分析器,没有借助正则表达式或者Lex之类的工具,就生生的写出来了。
- 在token流中,如何识别一个短语,或者说找到一个特定结构。这个过程叫解析或者句法分析。
- 用句法图来表示句法规则。句法图很形象的展现了哪些语句是合法的,哪些是非法的。
- 如何跟业界接轨,用上下文无关语法,或者BNF的技术来准确的描述一个编程语言的句法。
- 怎么把语法转换成代码,怎么写一个自顶向下递归调用的解析器。
- 怎么写一个解释器。
- 结合律和优先率。如何把用优先率表格来构建语法。
- 构建AST,来表示整个Pascal程序。
- 用一系列的visit_方法遍历AST。
我们的只是储备已经不少了,可以开始解决下面这些问题了:
- 嵌套过程和函数。
- 过程和函数调用。
- 语义分析,包括类型检查等等。
- 控制流,比如if语句。
- 数据类型的聚合,比如Records记录。
- 更多的内部类型。
- 代码调试器。
- 其他。
万里长征始于足下。我们需要打好基础,才能向往诗和远方。像符号、符号表、范畴等等的概念都值得再看一遍。
首先,什么是符号?我们定义符号就是一个程序中实体(比如变量、子程序、内部类型)的标识符。为了让符号有意义,它至少要包含下面三个东西:
- Name ( ‘x’, ‘y’, ‘number’)
- Category (variable, subroutine, or built-in type)
- Type (INTEGER, REAL)
今天我们只打算讲讲variable symbols 和 built-in type symbols。
下面Pascal程序有四个符号,两个variable symbols, x和y;两个built-in类型的symbols,INTEGER 和REAL。
我们创造一个Symbol 基础类:
class Symbol(object):
def __init__(self, name, type=None):
self.name = name
self.type = type
很奇怪,并没有category这个参数。因为我们会把category的信息体现在类名中。
比如:
class BuiltinTypeSymbol(Symbol):
def __init__(self, name):
super(BuiltinTypeSymbol, self).__init__(name)
def __str__(self):
return self.name
__repr__ = __str__
有两个奇怪名字的方法: __str__ and __repr__ 。它们是Python两个特殊的方法。我们已经把他们定义成了打印symbol对象良好排版格式的方法。
下载 interpreter file 保存成spi.py运行一下吧。完整的代码在文章最后面。
$ python >>> from spi import BuiltinTypeSymbol >>> int_type = BuiltinTypeSymbol('INTEGER') >>> int_type INTEGER >>> real_type = BuiltinTypeSymbol('REAL') >>> real_type REAL
类似的道理,variable symbol对应着 VarSymbol 类:
class VarSymbol(Symbol):
def __init__(self, name, type):
super(VarSymbol, self).__init__(name, type)
def __str__(self):
return '<{name}:{type}>'.format(name=self.name, type=self.type)
__repr__ = __str__
执行一下看看。变量的特别之处在于,它有点复杂了,看看那个尖括号,包含着名字和类型。
$ python >>> from spi import BuiltinTypeSymbol, VarSymbol >>> int_type = BuiltinTypeSymbol('INTEGER') >>> real_type = BuiltinTypeSymbol('REAL') >>> >>> var_x_symbol = VarSymbol('x', int_type) >>> var_x_symbol <x:INTEGER> >>> var_y_symbol = VarSymbol('y', real_type) >>> var_y_symbol <y:REAL>
Symbol的上下级关系是:
为什么我们要这么在乎symbol呢?因为我们要保证赋值给正确类型的symbol,变量必须事先声明,然后再使用。
比如下面的Pascal程序就是错误的,它炸了。用fpc编译一下就炸。把小数赋值给x,或者a没有声明。这些都是炸的原因。
为了避免炸,我们在运行时要构建一个小本本——symbol table来记录symbol。Symbol table严格的定义是,它是追踪源码中的各种symbol的ADT抽象数据类型。
class SymbolTable(object):
def __init__(self):
self._symbols = OrderedDict()
def __str__(self):
s = 'Symbols: {symbols}'.format(
symbols=[value for value in self._symbols.values()]
)
return s
__repr__ = __str__
def define(self, symbol):
print('Define: %s' % symbol)
self._symbols[symbol.name] = symbol
def lookup(self, name):
print('Lookup: %s' % name)
symbol = self._symbols.get(name)
# 'symbol' is either an instance of the Symbol class or 'None'
return symbol
为了存储和查找symbol table,我们定义了两个方法define和 lookup.
define 中的 _symbol 是一个排序字典,存放了symbol的名字和本身作为键值对。 lookup负责在字典中查找。
还记得我们的Pascal源代码吗?
PROGRAM Part11;
VAR
x : INTEGER;
y : REAL;
BEGIN
END.
我们可以在Python中试试新的符号表。
$ python
>>> from spi import SymbolTable, BuiltinTypeSymbol, VarSymbol
>>> symtab = SymbolTable()
>>> int_type = BuiltinTypeSymbol('INTEGER')
>>> symtab.define(int_type)
Define: INTEGER
>>> symtab
Symbols: [INTEGER]
>>>
>>> var_x_symbol = VarSymbol('x', int_type)
>>> symtab.define(var_x_symbol)
Define: <x:INTEGER>
>>> symtab
Symbols: [INTEGER, <x:INTEGER>]
>>>
>>> real_type = BuiltinTypeSymbol('REAL')
>>> symtab.define(real_type)
Define: REAL
>>> symtab
Symbols: [INTEGER, <x:INTEGER>, REAL]
>>>
>>> var_y_symbol = VarSymbol('y', real_type)
>>> symtab.define(var_y_symbol)
Define: <y:REAL>
>>> symtab
Symbols: [INTEGER, <x:INTEGER>, REAL, <y:REAL>]
_symbols 字典结构是这样的:
怎么才能让构建symbol table的过程自动化呢?我们需要另一个节点的visit方法来遍历AST。你可以看到AST的作用确实非凡,我们不需要扩展我们的parser来处理符号表,我们只需要再写一个节点的visit类就可以了。干净又漂亮!
首先,我们先扩展一下SymbolTable 类,让他初始化一下内部类型。
class SymbolTable(object):
def __init__(self):
self._symbols = OrderedDict()
self._init_builtins()
def _init_builtins(self):
self.define(BuiltinTypeSymbol('INTEGER'))
self.define(BuiltinTypeSymbol('REAL'))
def __str__(self):
s = 'Symbols: {symbols}'.format(
symbols=[value for value in self._symbols.values()]
)
return s
__repr__ = __str__
def define(self, symbol):
print('Define: %s' % symbol)
self._symbols[symbol.name] = symbol
def lookup(self, name):
print('Lookup: %s' % name)
symbol = self._symbols.get(name)
# 'symbol' is either an instance of the Symbol class or 'None'
return symbol
SymbolTableBuilder AST node visitor方法:
class SymbolTableBuilder(NodeVisitor):
def __init__(self):
self.symtab = SymbolTable()
def visit_Block(self, node):
for declaration in node.declarations:
self.visit(declaration)
self.visit(node.compound_statement)
def visit_Program(self, node):
self.visit(node.block)
def visit_BinOp(self, node):
self.visit(node.left)
self.visit(node.right)
def visit_Num(self, node):
pass
def visit_UnaryOp(self, node):
self.visit(node.expr)
def visit_Compound(self, node):
for child in node.children:
self.visit(child)
def visit_NoOp(self, node):
pass
def visit_VarDecl(self, node):
type_name = node.type_node.value
type_symbol = self.symtab.lookup(type_name)
var_name = node.var_node.value
var_symbol = VarSymbol(var_name, type_symbol)
self.symtab.define(var_symbol)
最后一个visit方法负责遍历VarDecl AST 节点,存储相应的symbol。首先这个方法在table中查找symbol的名字,然后创建一个varsymbol,最后存(定义)起来。
用Python命令行,演示一下SymbolTableBuilder AST 遍历的过程:
$ python
>>> from spi import Lexer, Parser, SymbolTableBuilder
>>> text = """
... PROGRAM Part11;
... VAR
... x : INTEGER;
... y : REAL;
...
... BEGIN
...
... END.
... """
>>> lexer = Lexer(text)
>>> parser = Parser(lexer)
>>> tree = parser.parse()
>>> symtab_builder = SymbolTableBuilder()
Define: INTEGER
Define: REAL
>>> symtab_builder.visit(tree)
Lookup: INTEGER
Define: <x:INTEGER>
Lookup: REAL
Define: <y:REAL>
>>> # Let’s examine the contents of our symbol table
…
>>> symtab_builder.symtab
Symbols: [INTEGER, REAL, <x:INTEGER>, <y:REAL>]
同时我们的代码具备了一种能力,验证变量的类型是否合法。我们增加了两个新的方法: visit_Assign 和 visit_Var:
def visit_Assign(self, node):
var_name = node.left.value
var_symbol = self.symtab.lookup(var_name)
if var_symbol is None:
raise NameError(repr(var_name))
self.visit(node.right)
def visit_Var(self, node):
var_name = node.value
var_symbol = self.symtab.lookup(var_name)
if var_symbol is None:
raise NameError(repr(var_name))
里面包含了异常的处理。比如下面这段错误的代码示范:
PROGRAM NameError1;
VAR
a : INTEGER;
BEGIN
a := 2 + b;
END.
让我们试试,如果我们为整个程序构建一个AST,然后扔给symbol table builder会怎么样?
$ python
>>> from spi import Lexer, Parser, SymbolTableBuilder
>>> text = """
... PROGRAM NameError1;
... VAR
... a : INTEGER;
...
... BEGIN
... a := 2 + b;
... END.
... """
>>> lexer = Lexer(text)
>>> parser = Parser(lexer)
>>> tree = parser.parse()
>>> symtab_builder = SymbolTableBuilder()
Define: INTEGER
Define: REAL
>>> symtab_builder.visit(tree)
Lookup: INTEGER
Define: <a:INTEGER>
Lookup: a
Lookup: b
Traceback (most recent call last):
...
File "spi.py", line 674, in visit_Var
raise NameError(repr(var_name))
NameError: 'b'
你看,a能顺利找到,b会出错。
另一种错误的情况是a没有定义就使用了:
PROGRAM NameError2;
VAR
b : INTEGER;
BEGIN
b := 1;
a := b + 2;
END.
Python命令行里面也可以验证这一点:
>>> from spi import Lexer, Parser, SymbolTableBuilder
>>> text = """
... PROGRAM NameError2;
... VAR
... b : INTEGER;
...
... BEGIN
... b := 1;
... a := b + 2;
... END.
... """
>>> lexer = Lexer(text)
>>> parser = Parser(lexer)
>>> tree = parser.parse()
>>> symtab_builder = SymbolTableBuilder()
Define: INTEGER
Define: REAL
>>> symtab_builder.visit(tree)
Lookup: INTEGER
Define: <b:INTEGER>
Lookup: b
Lookup: a
Traceback (most recent call last):
...
File "spi.py", line 665, in visit_Assign
raise NameError(repr(var_name))
NameError: 'a'
SymbolTableBuilder AST visitor 在运行时完成了检查,也就是interpreter真正计算之前。
比如这个程序:
PROGRAM Part11;
VAR
x : INTEGER;
BEGIN
x := 2;
END.
Symbol table和运行时的GLOBAL_MEMORY 在程序退出前是这样样子的:
看到没有,symbol table里面完全没有x=2的影子,因为interpreter完全“掌握了局势”。
第九章里面提到的symbol table还记得吗?
现在这种设计已经没有了!我们不在需要两份symbol table的拷贝了。
让我们完整的测试下面的程序:
PROGRAM Part11;
VAR
number : INTEGER;
a, b : INTEGER;
y : REAL;
BEGIN {Part11}
number := 2;
a := number ;
b := 10 * a + 10 * number DIV 4;
y := 20 / 7 + 3.14
END. {Part11}
运行看看:
$ python spi.py part11.pas
Define: INTEGER
Define: REAL
Lookup: INTEGER
Define: <number:INTEGER>
Lookup: INTEGER
Define: <a:INTEGER>
Lookup: INTEGER
Define: <b:INTEGER>
Lookup: REAL
Define: <y:REAL>
Lookup: number
Lookup: a
Lookup: number
Lookup: b
Lookup: a
Lookup: number
Lookup: y
Symbol Table contents:
Symbols: [INTEGER, REAL, <number:INTEGER>, <a:INTEGER>, <b:INTEGER>, <y:REAL>]
Run-time GLOBAL_MEMORY contents:
a = 2
b = 25
number = 2
y = 5.99714285714
我还想多罗嗦一句: Interpreter 类跟构建symbol table一点关系都没有,它只是调用一下SymbolTableBuilder 来确保代码中的变量在被interpreter使用前都被声明过了。
附录:interpreter完整代码:
""" SPI - Simple Pascal Interpreter. Part 11."""
from collections import OrderedDict
###############################################################################
# #
# LEXER #
# #
###############################################################################
# Token types
#
# EOF (end-of-file) token is used to indicate that
# there is no more input left for lexical analysis
INTEGER = 'INTEGER'
REAL = 'REAL'
INTEGER_CONST = 'INTEGER_CONST'
REAL_CONST = 'REAL_CONST'
PLUS = 'PLUS'
MINUS = 'MINUS'
MUL = 'MUL'
INTEGER_DIV = 'INTEGER_DIV'
FLOAT_DIV = 'FLOAT_DIV'
LPAREN = 'LPAREN'
RPAREN = 'RPAREN'
ID = 'ID'
ASSIGN = 'ASSIGN'
BEGIN = 'BEGIN'
END = 'END'
SEMI = 'SEMI'
DOT = 'DOT'
PROGRAM = 'PROGRAM'
VAR = 'VAR'
COLON = 'COLON'
COMMA = 'COMMA'
EOF = 'EOF'
class Token(object):
def __init__(self, type, value):
self.type = type
self.value = value
def __str__(self):
"""String representation of the class instance.
Examples:
Token(INTEGER, 3)
Token(PLUS, '+')
Token(MUL, '*')
"""
return 'Token({type}, {value})'.format(
type=self.type,
value=repr(self.value)
)
def __repr__(self):
return self.__str__()
RESERVED_KEYWORDS = {
'PROGRAM': Token('PROGRAM', 'PROGRAM'),
'VAR': Token('VAR', 'VAR'),
'DIV': Token('INTEGER_DIV', 'DIV'),
'INTEGER': Token('INTEGER', 'INTEGER'),
'REAL': Token('REAL', 'REAL'),
'BEGIN': Token('BEGIN', 'BEGIN'),
'END': Token('END', 'END'),
}
class Lexer(object):
def __init__(self, text):
# client string input, e.g. "4 + 2 * 3 - 6 / 2"
self.text = text
# self.pos is an index into self.text
self.pos = 0
self.current_char = self.text[self.pos]
def error(self):
raise Exception('Invalid character')
def advance(self):
"""Advance the `pos` pointer and set the `current_char` variable."""
self.pos += 1
if self.pos > len(self.text) - 1:
self.current_char = None # Indicates end of input
else:
self.current_char = self.text[self.pos]
def peek(self):
peek_pos = self.pos + 1
if peek_pos > len(self.text) - 1:
return None
else:
return self.text[peek_pos]
def skip_whitespace(self):
while self.current_char is not None and self.current_char.isspace():
self.advance()
def skip_comment(self):
while self.current_char != '}':
self.advance()
self.advance() # the closing curly brace
def number(self):
"""Return a (multidigit) integer or float consumed from the input."""
result = ''
while self.current_char is not None and self.current_char.isdigit():
result += self.current_char
self.advance()
if self.current_char == '.':
result += self.current_char
self.advance()
while (
self.current_char is not None and
self.current_char.isdigit()
):
result += self.current_char
self.advance()
token = Token('REAL_CONST', float(result))
else:
token = Token('INTEGER_CONST', int(result))
return token
def _id(self):
"""Handle identifiers and reserved keywords"""
result = ''
while self.current_char is not None and self.current_char.isalnum():
result += self.current_char
self.advance()
token = RESERVED_KEYWORDS.get(result, Token(ID, result))
return token
def get_next_token(self):
"""Lexical analyzer (also known as scanner or tokenizer)
This method is responsible for breaking a sentence
apart into tokens. One token at a time.
"""
while self.current_char is not None:
if self.current_char.isspace():
self.skip_whitespace()
continue
if self.current_char == '{':
self.advance()
self.skip_comment()
continue
if self.current_char.isalpha():
return self._id()
if self.current_char.isdigit():
return self.number()
if self.current_char == ':' and self.peek() == '=':
self.advance()
self.advance()
return Token(ASSIGN, ':=')
if self.current_char == ';':
self.advance()
return Token(SEMI, ';')
if self.current_char == ':':
self.advance()
return Token(COLON, ':')
if self.current_char == ',':
self.advance()
return Token(COMMA, ',')
if self.current_char == '+':
self.advance()
return Token(PLUS, '+')
if self.current_char == '-':
self.advance()
return Token(MINUS, '-')
if self.current_char == '*':
self.advance()
return Token(MUL, '*')
if self.current_char == '/':
self.advance()
return Token(FLOAT_DIV, '/')
if self.current_char == '(':
self.advance()
return Token(LPAREN, '(')
if self.current_char == ')':
self.advance()
return Token(RPAREN, ')')
if self.current_char == '.':
self.advance()
return Token(DOT, '.')
self.error()
return Token(EOF, None)
###############################################################################
# #
# PARSER #
# #
###############################################################################
class AST(object):
pass
class BinOp(AST):
def __init__(self, left, op, right):
self.left = left
self.token = self.op = op
self.right = right
class Num(AST):
def __init__(self, token):
self.token = token
self.value = token.value
class UnaryOp(AST):
def __init__(self, op, expr):
self.token = self.op = op
self.expr = expr
class Compound(AST):
"""Represents a 'BEGIN ... END' block"""
def __init__(self):
self.children = []
class Assign(AST):
def __init__(self, left, op, right):
self.left = left
self.token = self.op = op
self.right = right
class Var(AST):
"""The Var node is constructed out of ID token."""
def __init__(self, token):
self.token = token
self.value = token.value
class NoOp(AST):
pass
class Program(AST):
def __init__(self, name, block):
self.name = name
self.block = block
class Block(AST):
def __init__(self, declarations, compound_statement):
self.declarations = declarations
self.compound_statement = compound_statement
class VarDecl(AST):
def __init__(self, var_node, type_node):
self.var_node = var_node
self.type_node = type_node
class Type(AST):
def __init__(self, token):
self.token = token
self.value = token.value
class Parser(object):
def __init__(self, lexer):
self.lexer = lexer
# set current token to the first token taken from the input
self.current_token = self.lexer.get_next_token()
def error(self):
raise Exception('Invalid syntax')
def eat(self, token_type):
# compare the current token type with the passed token
# type and if they match then "eat" the current token
# and assign the next token to the self.current_token,
# otherwise raise an exception.
if self.current_token.type == token_type:
self.current_token = self.lexer.get_next_token()
else:
self.error()
def program(self):
"""program : PROGRAM variable SEMI block DOT"""
self.eat(PROGRAM)
var_node = self.variable()
prog_name = var_node.value
self.eat(SEMI)
block_node = self.block()
program_node = Program(prog_name, block_node)
self.eat(DOT)
return program_node
def block(self):
"""block : declarations compound_statement"""
declaration_nodes = self.declarations()
compound_statement_node = self.compound_statement()
node = Block(declaration_nodes, compound_statement_node)
return node
def declarations(self):
"""declarations : VAR (variable_declaration SEMI)+
| empty
"""
declarations = []
if self.current_token.type == VAR:
self.eat(VAR)
while self.current_token.type == ID:
var_decl = self.variable_declaration()
declarations.extend(var_decl)
self.eat(SEMI)
return declarations
def variable_declaration(self):
"""variable_declaration : ID (COMMA ID)* COLON type_spec"""
var_nodes = [Var(self.current_token)] # first ID
self.eat(ID)
while self.current_token.type == COMMA:
self.eat(COMMA)
var_nodes.append(Var(self.current_token))
self.eat(ID)
self.eat(COLON)
type_node = self.type_spec()
var_declarations = [
VarDecl(var_node, type_node)
for var_node in var_nodes
]
return var_declarations
def type_spec(self):
"""type_spec : INTEGER
| REAL
"""
token = self.current_token
if self.current_token.type == INTEGER:
self.eat(INTEGER)
else:
self.eat(REAL)
node = Type(token)
return node
def compound_statement(self):
"""
compound_statement: BEGIN statement_list END
"""
self.eat(BEGIN)
nodes = self.statement_list()
self.eat(END)
root = Compound()
for node in nodes:
root.children.append(node)
return root
def statement_list(self):
"""
statement_list : statement
| statement SEMI statement_list
"""
node = self.statement()
results = [node]
while self.current_token.type == SEMI:
self.eat(SEMI)
results.append(self.statement())
return results
def statement(self):
"""
statement : compound_statement
| assignment_statement
| empty
"""
if self.current_token.type == BEGIN:
node = self.compound_statement()
elif self.current_token.type == ID:
node = self.assignment_statement()
else:
node = self.empty()
return node
def assignment_statement(self):
"""
assignment_statement : variable ASSIGN expr
"""
left = self.variable()
token = self.current_token
self.eat(ASSIGN)
right = self.expr()
node = Assign(left, token, right)
return node
def variable(self):
"""
variable : ID
"""
node = Var(self.current_token)
self.eat(ID)
return node
def empty(self):
"""An empty production"""
return NoOp()
def expr(self):
"""
expr : term ((PLUS | MINUS) term)*
"""
node = self.term()
while self.current_token.type in (PLUS, MINUS):
token = self.current_token
if token.type == PLUS:
self.eat(PLUS)
elif token.type == MINUS:
self.eat(MINUS)
node = BinOp(left=node, op=token, right=self.term())
return node
def term(self):
"""term : factor ((MUL | INTEGER_DIV | FLOAT_DIV) factor)*"""
node = self.factor()
while self.current_token.type in (MUL, INTEGER_DIV, FLOAT_DIV):
token = self.current_token
if token.type == MUL:
self.eat(MUL)
elif token.type == INTEGER_DIV:
self.eat(INTEGER_DIV)
elif token.type == FLOAT_DIV:
self.eat(FLOAT_DIV)
node = BinOp(left=node, op=token, right=self.factor())
return node
def factor(self):
"""factor : PLUS factor
| MINUS factor
| INTEGER_CONST
| REAL_CONST
| LPAREN expr RPAREN
| variable
"""
token = self.current_token
if token.type == PLUS:
self.eat(PLUS)
node = UnaryOp(token, self.factor())
return node
elif token.type == MINUS:
self.eat(MINUS)
node = UnaryOp(token, self.factor())
return node
elif token.type == INTEGER_CONST:
self.eat(INTEGER_CONST)
return Num(token)
elif token.type == REAL_CONST:
self.eat(REAL_CONST)
return Num(token)
elif token.type == LPAREN:
self.eat(LPAREN)
node = self.expr()
self.eat(RPAREN)
return node
else:
node = self.variable()
return node
def parse(self):
"""
program : PROGRAM variable SEMI block DOT
block : declarations compound_statement
declarations : VAR (variable_declaration SEMI)+
| empty
variable_declaration : ID (COMMA ID)* COLON type_spec
type_spec : INTEGER
compound_statement : BEGIN statement_list END
statement_list : statement
| statement SEMI statement_list
statement : compound_statement
| assignment_statement
| empty
assignment_statement : variable ASSIGN expr
empty :
expr : term ((PLUS | MINUS) term)*
term : factor ((MUL | INTEGER_DIV | FLOAT_DIV) factor)*
factor : PLUS factor
| MINUS factor
| INTEGER_CONST
| REAL_CONST
| LPAREN expr RPAREN
| variable
variable: ID
"""
node = self.program()
if self.current_token.type != EOF:
self.error()
return node
###############################################################################
# #
# AST visitors (walkers) #
# #
###############################################################################
class NodeVisitor(object):
def visit(self, node):
method_name = 'visit_' + type(node).__name__
visitor = getattr(self, method_name, self.generic_visit)
return visitor(node)
def generic_visit(self, node):
raise Exception('No visit_{} method'.format(type(node).__name__))
###############################################################################
# #
# SYMBOLS and SYMBOL TABLE #
# #
###############################################################################
class Symbol(object):
def __init__(self, name, type=None):
self.name = name
self.type = type
class VarSymbol(Symbol):
def __init__(self, name, type):
super(VarSymbol, self).__init__(name, type)
def __str__(self):
return '<{name}:{type}>'.format(name=self.name, type=self.type)
__repr__ = __str__
class BuiltinTypeSymbol(Symbol):
def __init__(self, name):
super(BuiltinTypeSymbol, self).__init__(name)
def __str__(self):
return self.name
__repr__ = __str__
class SymbolTable(object):
def __init__(self):
self._symbols = OrderedDict()
self._init_builtins()
def _init_builtins(self):
self.define(BuiltinTypeSymbol('INTEGER'))
self.define(BuiltinTypeSymbol('REAL'))
def __str__(self):
s = 'Symbols: {symbols}'.format(
symbols=[value for value in self._symbols.values()]
)
return s
__repr__ = __str__
def define(self, symbol):
print('Define: %s' % symbol)
self._symbols[symbol.name] = symbol
def lookup(self, name):
print('Lookup: %s' % name)
symbol = self._symbols.get(name)
# 'symbol' is either an instance of the Symbol class or 'None'
return symbol
class SymbolTableBuilder(NodeVisitor):
def __init__(self):
self.symtab = SymbolTable()
def visit_Block(self, node):
for declaration in node.declarations:
self.visit(declaration)
self.visit(node.compound_statement)
def visit_Program(self, node):
self.visit(node.block)
def visit_BinOp(self, node):
self.visit(node.left)
self.visit(node.right)
def visit_Num(self, node):
pass
def visit_UnaryOp(self, node):
self.visit(node.expr)
def visit_Compound(self, node):
for child in node.children:
self.visit(child)
def visit_NoOp(self, node):
pass
def visit_VarDecl(self, node):
type_name = node.type_node.value
type_symbol = self.symtab.lookup(type_name)
var_name = node.var_node.value
var_symbol = VarSymbol(var_name, type_symbol)
self.symtab.define(var_symbol)
def visit_Assign(self, node):
var_name = node.left.value
var_symbol = self.symtab.lookup(var_name)
if var_symbol is None:
raise NameError(repr(var_name))
self.visit(node.right)
def visit_Var(self, node):
var_name = node.value
var_symbol = self.symtab.lookup(var_name)
if var_symbol is None:
raise NameError(repr(var_name))
###############################################################################
# #
# INTERPRETER #
# #
###############################################################################
class Interpreter(NodeVisitor):
def __init__(self, tree):
self.tree = tree
self.GLOBAL_MEMORY = OrderedDict()
def visit_Program(self, node):
self.visit(node.block)
def visit_Block(self, node):
for declaration in node.declarations:
self.visit(declaration)
self.visit(node.compound_statement)
def visit_VarDecl(self, node):
# Do nothing
pass
def visit_Type(self, node):
# Do nothing
pass
def visit_BinOp(self, node):
if node.op.type == PLUS:
return self.visit(node.left) + self.visit(node.right)
elif node.op.type == MINUS:
return self.visit(node.left) - self.visit(node.right)
elif node.op.type == MUL:
return self.visit(node.left) * self.visit(node.right)
elif node.op.type == INTEGER_DIV:
return self.visit(node.left) // self.visit(node.right)
elif node.op.type == FLOAT_DIV:
return float(self.visit(node.left)) / float(self.visit(node.right))
def visit_Num(self, node):
return node.value
def visit_UnaryOp(self, node):
op = node.op.type
if op == PLUS:
return +self.visit(node.expr)
elif op == MINUS:
return -self.visit(node.expr)
def visit_Compound(self, node):
for child in node.children:
self.visit(child)
def visit_Assign(self, node):
var_name = node.left.value
var_value = self.visit(node.right)
self.GLOBAL_MEMORY[var_name] = var_value
def visit_Var(self, node):
var_name = node.value
var_value = self.GLOBAL_MEMORY.get(var_name)
return var_value
def visit_NoOp(self, node):
pass
def interpret(self):
tree = self.tree
if tree is None:
return ''
return self.visit(tree)
def main():
import sys
text = open(sys.argv[1], 'r').read()
lexer = Lexer(text)
parser = Parser(lexer)
tree = parser.parse()
symtab_builder = SymbolTableBuilder()
symtab_builder.visit(tree)
print('')
print('Symbol Table contents:')
print(symtab_builder.symtab)
interpreter = Interpreter(tree)
result = interpreter.interpret()
print('')
print('Run-time GLOBAL_MEMORY contents:')
for k, v in sorted(interpreter.GLOBAL_MEMORY.items()):
print('%s = %s' % (k, v))
if __name__ == '__main__':
main()