Gieno Monadic Parser in C# | ||
Parser combinators are an idea that I enjoy every time I go back and look at again. They are an approach to building parsers by composing very simple atomic parsers into bigger and bigger units which can ultimately express real world grammars. This idea has been particularly popular in functional languages where the parsers can naturally be thought of as functions from input strings to parse trees, and composition of parsers is just function composition. This approach often leads to a simple syntax which makes the resulting parsers pleasantly declarative in that internal-DSL kind of way. Building a Parser Combinator Framework in C#The code below is some sample code I wrote shortly after getting ahold of one of the first C# prototype compilers a couple years ago. It uses some of the new languages features in interestingly unusual ways, which I'll highlight as I go along. Step 1: A Type for Representing Parsersusing System; using System.Linq; using System.Text; // The result of a parse consists of a value and the unconsumed input. public class Result<TInput, TValue> { public readonly TValue Value; public readonly TInput Rest; public Result(TValue value, TInput rest) { Value = value; Rest = rest; } } // A Parser is a delegate which takes an input and returns a result. public delegate Result<TInput, TValue> Parser<TInput, TValue>(TInput input); Taking a queue from the functional language implementations, our type for representing parsers will be a delegate type. This naturally captures the idea that a parser is a function which takes in some input, and returns a value as well as the unconsumed part of the input. Step 2: Provide Infix AND and OR Operations on Parserspublic static class ParserCombinatorExtensions { public static Parser<TInput, TValue> OR<TInput, TValue>( Before building any actual parsers, we'll just come up with ways to combine them. There are two obvious ways. Try the first one, and if it fails, apply the second one. This corresponds to alternate productions in a BNF grammar. Or we could apply the first one, and then apply the second one to whatever input is left. This corresponds to sequencing (juxtaposition) in BNF grammars. Note that these are extension methods applied to the Parser type, which is a delegate. This is something we couldn't have done in C# prior to C# 3.0, because there was no way to add methods to a delegate type. The "infix" syntax we get by making these extension methods instead of plain static methods is very important for the readability of the resulting parsers, where ordering is critical, as the example at the end of this post will show. Step 3: Support Composing Parsers using C# Query Expressionspublic static class ParserCombinatorsMonad { // By providing Select, Where and SelectMany methods on Parser<TInput,TValue> we make the // C# Query Expression syntax available for manipulating Parsers. public static Parser<TInput, TValue> Where<TInput, TValue>( This is where the fun really starts! Any type which implements Select, SelectMany and Where methods supports (part of) the "query pattern" which means we can write C#3.0 queries including multiple froms, an optional where clause and a select clause to process objects of this type. This can make it a *lot* easier to combine parsers, and makes the syntax more transparent. That the parsers we are using here support these three operations is captured in the idea that this parser type is a monad - something very much in LINQ's blood - which I hope to write more about in a future post. Step 4: The Fundamental Parser Combinator Building Blocks// Contains all the basic parsers that are independent of return type. public abstract class Parsers<TInput> { public Parser<TInput, TValue> Succeed<TValue>(TValue value) { return input => new Result<TInput, TValue>(value, input); } public Parser<TInput, TValue[]> Rep<TValue>(Parser<TInput, TValue> parser) { return Rep1(parser).OR(Succeed(new TValue[0])); } public Parser<TInput, TValue[]> Rep1<TValue>(Parser<TInput, TValue> parser) { return from x in parser from xs in Rep(parser) select (new[] { x }).Concat(xs).ToArray(); } } // Contains a few parsers that parse characters from an input stream public abstract class CharParsers<TInput> : Parsers<TInput> { public abstract Parser<TInput, char> AnyChar { get; } public Parser<TInput, char> Char(char ch) { So now we need to write some (almost) real parsers. The Succeed parser always succeeds with the provided result without consuming any input. Rep and Rep1 apply a parser repeatedly, they differ only in whether they require the parser to parse at least once. Note that Rep1 uses Query Expressions to capture the idea of parsing using "parser", then parsing using "Rep(parser)" then returning a result built out of the two individual results. The AnyChar and Char parsers parse actual characters out of the input stream. AnyChar parses any single character, whereas the two overloads of Char only accept certain classes of characters. Note that AnyChar is defined abstract because it can only be implemented once we've decided what kind of input we're going to parse - but we want to be able to build on it without committing to a particular input type. Step 5: An Example Parser for a MiniML Language// Term and its derived classes define the AST for terms in the MiniML language. public abstract class Term { } public class LambdaTerm : Term { public readonly string Ident; Here we've used the parsers built so far to implemented a parser for a (toy) language called MiniML. You can immediately see that the parser is structured much like the BNF it implements - which is the internal-DSL concept I mentioned above. You can also see how C# Query Expressions are used to help with this - in a context that is very different than their standard query usage. Step 6: Parse Something!public class MiniMLParserFromString: MiniMLParsers<string> { public override Parser<string, char> AnyChar { get { { return input => input.Length > 0 ? new Result<string,char>(input[0], input.Substring(1)) : null; } } } }class Program { static void Main(string[] args) { MiniMLParserFromString parser = new MiniMLParserFromString(); Result<string,Term> result = parser.All(@"let true = \x.\y.x in let false = \x.\y.y in let if = \b.\l.\r.(b l) r in if true then false else true;"); } } We finally have something we can execute! MiniMLParserFromString makes everything concrete by describing how to parse characters from a string input. And then in the Main method we see an example which parsers a little example expression from the MiniML language.
| ||
转载于:https://www.cnblogs.com/gieno/archive/2009/09/09/1563063.html