LL parser

LL parser

From Wikipedia, the free encyclopedia

In computer science, an LL parser is a top-down parser for a subset of the context-free grammars. It parses the input from Left to right, and constructs a Leftmost derivation of the sentence (hence LL, compared with LR parser). The class of grammars which are parsable in this way is known as the LL grammars.

The remainder of this article describes the table-based kind of parser, the alternative being a recursive descent parser which is usually coded by hand (although not always; see e.g. ANTLR for an LL(*) recursive-descent parser generator).

An LL parser is called an LL(k) parser if it uses k tokens of lookahead when parsing a sentence. If such a parser exists for a certain grammar and it can parse sentences of this grammar without backtracking then it is called an LL(k) grammar. A language that has an LL(k) grammar is known as an LL(k) language. There are LL(k+n) languages that are not LL(k) languages.[1] A corollary of this is that not all context-free languages are LL(k) languages.

LL(1) grammars are very popular because the corresponding LL parsers only need to look at the next token to make their parsing decisions. Languages based on grammars with a high value of k have traditionally been considered[who?] to be difficult to parse, although this is less true now given the availability and widespread use[citation needed] of parser generators supporting LL(k) grammars for arbitrary k.

An LL parser is called an LL(*) parser if it is not restricted to a finite k tokens of lookahead, but can make parsing decisions by recognizing whether the following tokens belong to a regular language (for example by use of a Deterministic Finite Automaton).

Contents

   [hide

[edit]General case

The parser works on strings from a particular context-free grammar.

The parser consists of

  • an input buffer, holding the input string (built from the grammar)
  • stack on which to store the terminals and non-terminals from the grammar yet to be parsed
  • parsing table which tells it what (if any) grammar rule to apply given the symbols on top of its stack and the next input token

The parser applies the rule found in the table by matching the top-most symbol on the stack (row) with the current symbol in the input stream (column).

When the parser starts, the stack already contains two symbols:

[ S, $ ]

where '$' is a special terminal to indicate the bottom of the stack and the end of the input stream, and 'S' is the start symbol of the grammar. The parser will attempt to rewrite the contents of this stack to what it sees on the input stream. However, it only keeps on the stack what still needs to be rewritten.

[edit]Concrete example

[edit]Set up

To explain its workings we will consider the following small grammar:

  1. S → F
  2. S → ( S + F )
  3. F → a

and parse the following input:

( a + a )

The parsing table for this grammar looks as follows:

  ( ) a + $
S 2 - 1 - -
F - - 3 - -

(Note that there is also a column for the special terminal, represented here as $, that is used to indicate the end of the input stream.)

[edit]Parsing procedure

In each step, the parser reads the next-available symbol from the input stream, and the top-most symbol from the stack. If the input symbol and the stack-top symbol match, the parser discards them both, leaving only the unmatched symbols in the input stream and on the stack.

Thus, in its first step, the parser reads the input symbol '(' and the stack-top symbol 'S'. The parsing table instruction comes from the column headed by the input symbol '(' and the row headed by the stack-top symbol 'S'; this cell contains '2', which instructs the parser to apply rule (2). The parser has to rewrite 'S' to '( S + F )' on the stack and write the rule number 2 to the output. The stack then becomes:

[ (, S, +, F, ), $ ]

Since the '(' from the input stream did not match the top-most symbol, 'S', from the stack, it was not removed, and remains the next-available input symbol for the following step.

In the second step, the parser removes the '(' from its input stream and from its stack, since they match. The stack now becomes:

[ S, +, F, ), $ ]

Now the parser has an 'a' on its input stream and an 'S' as its stack top. The parsing table instructs it to apply rule (1) from the grammar and write the rule number 1 to the output stream. The stack becomes:

[ F, +, F, ), $ ]

The parser now has an 'a' on its input stream and an 'F' as its stack top. The parsing table instructs it to apply rule (3) from the grammar and write the rule number 3 to the output stream. The stack becomes:

[ a, +, F, ), $ ]

In the next two steps the parser reads the 'a' and '+' from the input stream and, since they match the next two items on the stack, also removes them from the stack. This results in:

[ F, ), $ ]

In the next three steps the parser will replace 'F' on the stack by 'a', write the rule number 3 to the output stream and remove the 'a' and ')' from both the stack and the input stream. The parser thus ends with '$' on both its stack and its input stream.

In this case the parser will report that it has accepted the input string and write the following list of rule numbers to the output stream:

[ 2, 1, 3, 3 ]

This is indeed a list of rules for a leftmost derivation of the input string, which is:

S →  ( S  + F  ) →  ( F  + F  ) →  ( a + F  ) →  ( a + a )

[edit]Parser implementation in C++

Below follows a C++ implementation of a table-based LL parser for the example language:

#include <iostream>
#include <map>
#include <stack>
 
enum Symbols {
        // the symbols:
        // Terminal symbols:
        TS_L_PARENS,    // (
        TS_R_PARENS,    // )
        TS_A,           // a
        TS_PLUS,        // +
        TS_EOS,         // $, in this case corresponds to '\0'
        TS_INVALID,     // invalid token
 
        // Non-terminal symbols:
        NTS_S,          // S
        NTS_F
};
 
/* 
Converts a valid token to the corresponding terminal symbol
*/
enum Symbols lexer(char c)
{
        switch(c)
        {
                case '(':  return TS_L_PARENS;
                case ')':  return TS_R_PARENS;
                case 'a':  return TS_A;
                case '+':  return TS_PLUS;
                case '\0': return TS_EOS; // end of stack: the $ terminal symbol
                default:   return TS_INVALID;
        }
}
 
int main(int argc, char **argv)
{
        using namespace std;
 
        if (argc < 2)
        {
                cout << "usage:\n\tll '(a+a)'" << endl;
                return 0;
        }
 
        // LL parser table, maps < non-terminal, terminal> pair to action       
        map< enum Symbols, map<enum Symbols, int> > table; 
        stack<enum Symbols>     ss;     // symbol stack
        char *p;        // input buffer
 
        // initialize the symbols stack
        ss.push(TS_EOS);        // terminal, $
        ss.push(NTS_S);         // non-terminal, S
 
        // initialize the symbol stream cursor
        p = &argv[1][0];
 
        // setup the parsing table
        table[NTS_S][TS_L_PARENS] = 2;  table[NTS_S][TS_A] = 1;
        table[NTS_F][TS_A] = 3;
 
        while(ss.size() > 0)
        {
                if(lexer(*p) == ss.top())
                {
                        cout << "Matched symbols: " << lexer(*p) << endl;
                        p++;
                        ss.pop();
                }
                else
                {
                        cout << "Rule " << table[ss.top()][lexer(*p)] << endl;
                        switch(table[ss.top()][lexer(*p)])
                        {
                                case 1: // 1. S → F
                                        ss.pop();
                                        ss.push(NTS_F); // F
                                        break;
 
                                case 2: // 2. S → ( S + F )
                                        ss.pop();
                                        ss.push(TS_R_PARENS);   // )
                                        ss.push(NTS_F);         // F
                                        ss.push(TS_PLUS);       // +
                                        ss.push(NTS_S);         // S
                                        ss.push(TS_L_PARENS);   // (
                                        break;
 
                                case 3: // 3. F → a
                                        ss.pop();
                                        ss.push(TS_A);  // a
                                        break;
 
                                default:
                                        cout << "parsing table defaulted" << endl;
                                        return 0;
                                        break;
                        }
                }
        }
 
        cout << "finished parsing" << endl;
 
        return 0;
}

[edit]Remarks

As can be seen from the example the parser performs three types of steps depending on whether the top of the stack is a nonterminal, a terminal or the special symbol $:

  • If the top is a nonterminal then it looks up in the parsing table on the basis of this nonterminal and the symbol on the input stream which rule of the grammar it should use to replace it with on the stack. The number of the rule is written to the output stream. If the parsing table indicates that there is no such rule then it reports an error and stops.
  • If the top is a terminal then it compares it to the symbol on the input stream and if they are equal they are both removed. If they are not equal the parser reports an error and stops.
  • If the top is $ and on the input stream there is also a $ then the parser reports that it has successfully parsed the input, otherwise it reports an error. In both cases the parser will stop.

These steps are repeated until the parser stops, and then it will have either completely parsed the input and written a leftmost derivation to the output stream or it will have reported an error.

[edit]Constructing an LL(1) parsing table

In order to fill the parsing table, we have to establish what grammar rule the parser should choose if it sees a nonterminal A on the top of its stack and a symbol a on its input stream. It is easy to see that such a rule should be of the form A → w and that the language corresponding to w should have at least one string starting with a. For this purpose we define the First-set of w, written here as Fi(w), as the set of terminals that can be found at the start of some string in w, plus ε if the empty string also belongs to w. Given a grammar with the rules A1 → w1, ..., An → wn, we can compute theFi(wi) and Fi(Ai) for every rule as follows:

  1. initialize every Fi(wi) and Fi(Ai) with the empty set
  2. add Fi(wi) to Fi(Ai) for every rule Ai → wi, where Fi is defined as follows:
    • Fi(a w' ) = { a } for every terminal a
    • Fi(A w' ) = Fi(A) for every nonterminal A with ε not in Fi(A)
    • Fi(A w' ) = Fi(A) \ { ε } ∪ Fi(w' ) for every nonterminal A with ε in Fi(A)
    • Fi(ε) = { ε }
  3. add Fi(wi) to Fi(Ai) for every rule Ai → wi
  4. do steps 2 and 3 until all Fi sets stay the same.

Unfortunately, the First-sets are not sufficient to compute the parsing table. This is because a right-hand side w of a rule might ultimately be rewritten to the empty string. So the parser should also use the rule A → w if ε is in Fi(w) and it sees on the input stream a symbol that could followA. Therefore we also need the Follow-set of A, written as Fo(A) here, which is defined as the set of terminals a such that there is a string of symbolsαAaβ that can be derived from the start symbol. Computing the Follow-sets for the nonterminals in a grammar can be done as follows:

  1. initialize every Fo(Ai) with the empty set
  2. if there is a rule of the form Aj → wAiw' , then
    • if the terminal a is in Fi(w' ), then add a to Fo(Ai)
    • if ε is in Fi(w' ), then add Fo(Aj) to Fo(Ai)
  3. repeat step 2 until all Fo sets stay the same.

Now we can define exactly which rules will be contained where in the parsing table. If T[Aa] denotes the entry in the table for nonterminal A and terminal a, then

T[ A, a] contains the rule  A →  w if and only if
a is in  Fi( w) or
ε is in  Fi( w) and  a is in  Fo( A).

If the table contains at most one rule in every one of its cells, then the parser will always know which rule it has to use and can therefore parse strings without backtracking. It is in precisely this case that the grammar is called an LL(1) grammar.

[edit]Constructing an LL(k) parsing table

Until the mid 1990s, it was widely believed that LL(k) parsing (for k > 1) was impractical[citation needed], since the parse table would have exponentialsize in k in the worst case. This perception changed gradually after the release of the PCCTS around 1992, when it was demonstrated that many programming languages can be parsed efficiently by an LL(k) parser without triggering the worst-case behavior of the parser. Moreover, in certain cases LL parsing is feasible even with unlimited lookahead. By contrast, traditional parser generators like yacc use LALR(1) parse tables to construct a restricted LR parserwith a fixed one-token lookahead.

[edit]Conflicts

As described in the introduction, LL(1) parsers recognize languages that have LL(1) grammars, which are a special case of context-free grammars (CFG's); LL(1) parsers cannot recognize all context-free languages. The LL(1) languages are exactly those recognized by deterministic pushdown automata restricted to a single state [2]. In order for a CFG to be an LL(1) grammar, certain conflicts must not arise, which we describe in this section.

[edit]Terminology[3]

Let A be a non-terminal. FIRST(A) is (defined to be) the set of terminals that can appear in the first position of any string derived from A. FOLLOW(A) is the union over FIRST(B) where B is any non-terminal that immediately follows A in the right hand side of a production rule.

[edit]LL(1) Conflicts

There are 3 types of LL(1) conflicts:

  • FIRST/FIRST conflict
The FIRST sets of two different non-terminals intersect.
  • FIRST/FOLLOW conflict
The FIRST and FOLLOW set of a grammar rule overlap. With an epsilon in the FIRST set it is unknown which alternative to select.
An example of an LL(1) conflict:
 S -> A 'a' 'b'
 A -> 'a' | epsilon
The FIRST set of A now is { 'a' epsilon } and the FOLLOW set { 'a' }.
  • left-recursion
Left recursion will cause a FIRST/FIRST conflict with all alternatives.
 E -> E '+' term | alt1 | alt2

[edit]Solutions to LL(1) Conflicts

  • Left-factoring

A common left-factor is "factored out".

 A -> X | X Y Z

becomes

 A -> X B
 B -> Y Z | ε

Can be applied when two alternatives start with the same symbol like a FIRST/FIRST conflict.

  • Substitution

Substituting a rule into another rule to remove indirect or FIRST/FOLLOW conflicts. Note that this may cause a FIRST/FIRST conflict.

  • Left recursion removal[4]

A simple example for left recursion removal: The following production rule has left recursion on E

 E -> E '+' T
   -> T

This rule is nothing but list of T's separated by '+'. In a regular expression form T ('+' T)*. So the rule could be rewritten as

 E -> T Z
 Z -> '+' T Z
   -> ε

Now there is no left recursion and no conflicts on either of the rules.

However, not all CFGs have an equivalent LL(k)-grammar, e.g.:

 S -> A | B
 A -> 'a' A 'b' | ε
 B -> 'a' B 'b' 'b' | ε

It can be shown that there does not exist any LL(k)-grammar accepting the language generated by this grammar.

[edit]See also

[edit]External links

[edit]Notes

  1. ^ http://portal.acm.org/citation.cfm?id=805431
  2. ^ Kurki-Suonio, R. (1969). "Notes on top-down languages". BIT 9 (3): 225–238. doi:10.1007/BF01946814. edit
  3. ^ http://www.cs.uaf.edu/~cs331/notes/LL.pdf
  4. ^ Modern Compiler Design, Grune, Bal, Jacobs and Langendoen
  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
LL(1)文法是一种上下文无关文法,可以用于描述程序语言的语法结构。在编译原理中,LL(1)文法分析器是一种自顶向下的语法分析器,它可以根据输入的符号串,判断是否符合给定文法的规则,并构建出语法树。 在C语言中,可以使用LL(1)文法分析器来解析C源代码,以便进行语法检查、代码生成等操作。下面是一个简单的LL(1)文法分析器的实现,可以用来解析C语言的基本语法结构: 1. 定义文法规则 首先,需要定义C语言的文法规则。例如,可以定义如下的文法规则: ``` <program> ::= <declaration-list> <declaration-list> ::= <declaration> | <declaration> <declaration-list> <declaration> ::= <type-specifier> <identifier> ";" | <type-specifier> <identifier> "(" <parameter-list> ")" <compound-statement> <type-specifier> ::= "void" | "int" | "float" | "double" | "char" | "short" | "long" <identifier> ::= <letter> | <identifier> <letter> | <identifier> <digit> <letter> ::= "a" | "b" | "c" | ... | "z" | "A" | "B" | "C" | ... | "Z" <digit> ::= "0" | "1" | "2" | ... | "9" <parameter-list> ::= <parameter-declaration> | <parameter-declaration> "," <parameter-list> <parameter-declaration> ::= <type-specifier> <identifier> <compound-statement> ::= "{" <declaration-list> <statement-list> "}" <statement-list> ::= <statement> | <statement> <statement-list> <statement> ::= <expression-statement> | <compound-statement> | <selection-statement> | <iteration-statement> | <jump-statement> <expression-statement> ::= <expression> ";" <expression> ::= <assignment-expression> | <expression> "," <assignment-expression> <assignment-expression> ::= <conditional-expression> | <unary-expression> <assignment-operator> <assignment-expression> <conditional-expression> ::= <logical-or-expression> | <logical-or-expression> "?" <expression> ":" <conditional-expression> <logical-or-expression> ::= <logical-and-expression> | <logical-or-expression> "||" <logical-and-expression> <logical-and-expression> ::= <equality-expression> | <logical-and-expression> "&&" <equality-expression> <equality-expression> ::= <relational-expression> | <equality-expression> "==" <relational-expression> | <equality-expression> "!=" <relational-expression> <relational-expression> ::= <additive-expression> | <relational-expression> "<" <additive-expression> | <relational-expression> "<=" <additive-expression> | <relational-expression> ">" <additive-expression> | <relational-expression> ">=" <additive-expression> <additive-expression> ::= <multiplicative-expression> | <additive-expression> "+" <multiplicative-expression> | <additive-expression> "-" <multiplicative-expression> <multiplicative-expression> ::= <unary-expression> | <multiplicative-expression> "*" <unary-expression> | <multiplicative-expression> "/" <unary-expression> <unary-expression> ::= <primary-expression> | <unary-operator> <unary-expression> <primary-expression> ::= <identifier> | <constant> | "(" <expression> ")" <constant> ::= <integer-constant> | <floating-constant> | <character-constant> <integer-constant> ::= <digit> | <integer-constant> <digit> <floating-constant> ::= <digit> "." <digit> | <digit> "." <digit> <exponent-part> | "." <digit> <exponent-part> | <digit> <exponent-part> <exponent-part> ::= "e" <sign> <digit> | "E" <sign> <digit> | <empty> <sign> ::= "+" | "-" <character-constant> ::= "'" <character> "'" <character> ::= <letter> | <digit> | <special-character> <special-character> ::= "~" | "!" | "@" | "#" | "$" | "%" | "^" | "&" | "*" | "(" | ")" | "-" | "+" | "=" | "{" | "}" | "[" | "]" | ":" | ";" | "<" | ">" | "," | "." | "?" | "/" <unary-operator> ::= "+" | "-" | "!" | "~" <assignment-operator> ::= "=" | "+=" | "-=" | "*=" | "/=" <selection-statement> ::= "if" "(" <expression> ")" <statement> | "if" "(" <expression> ")" <statement> "else" <statement> | "switch" "(" <expression> ")" <compound-statement> <iteration-statement> ::= "while" "(" <expression> ")" <statement> | "do" <statement> "while" "(" <expression> ")" ";" | "for" "(" <expression> ";" <expression> ";" <expression> ")" <statement> | "for" "(" <declaration> <expression> ";" <expression> ")" <statement> <jump-statement> ::= "goto" <identifier> ";" | "continue" ";" | "break" ";" | "return" <expression> ";" | "return" ";" ``` 2. 实现LL(1)文法分析器 接下来,可以根据上述文法规则,实现LL(1)文法分析器。具体实现步骤如下: (1)将输入的符号串转换为Token序列,即将源代码中的每个单词转换为对应的Token类型。 (2)定义LL(1)分析表,该表包含所有的非终结符和终结符,以及它们之间的推导规则。根据文法规则,可以逐步构建出LL(1)分析表。 (3)根据LL(1)分析表和Token序列,进行语法分析。具体步骤如下: - 从Token序列中读入下一个Token,记为lookahead。 - 根据当前的非终结符和lookahead,查找LL(1)分析表中对应的推导规则。 - 如果推导规则中存在非终结符,则将该非终结符入栈,继续进行分析。 - 如果推导规则中存在终结符,则将该终结符与lookahead进行匹配,如果匹配成功,则弹出栈顶的非终结符,继续进行分析;否则,分析失败。 - 如果Token序列已经分析完毕,并且栈中只剩下一个非终结符(即开始符号),则分析成功;否则,分析失败。 下面是一个简单的LL(1)文法分析器的伪代码实现: ``` function LL1Parser(input): initialize stack with start symbol initialize lookahead with first token while stack is not empty: if top of stack is non-terminal: if LL(1) table contains (top of stack, lookahead): replace top of stack with corresponding production rule continue parsing else: report syntax error return else if top of stack is terminal: if top of stack matches lookahead: pop top of stack read next token from input and set it as lookahead continue parsing else: report syntax error return if input is fully parsed and stack is empty: report success else: report syntax error ``` 以上就是一个简单的LL(1)文法分析器的实现方法,可以用来解析C语言的基本语法结构。

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值