知乎周源微信_每周源代码59-开源宝藏：具有讽刺意味的.NET语言实现工具包

知乎周源微信

One of the best, if not the best way to sharpen the saw and keep your software development skills up to date is by reading code. Sure, write lots of code, but don't forget to explore other people's brains code. There's always fifteen different ways to create a "textboxes over data" application, and while it's interesting to take a look at whatever the newest way to make business software, sometimes it's nice to relax by looking at some implementations of classic software issues like parsers, lexers, and abstract syntax trees. If you didn't go to school or failed to take a compilers class at least knowing that this area of software engineering exists and is accessible to you is very important.

最好的方法之一(如果不是最好的话)可以使锯锐化并保持软件开发技能的最新状态，那就是阅读代码。 当然，可以编写很多代码，但不要忘记探索别人的 大脑 代码。 总是有十五种不同的方法来创建“基于数据的文本框”应用程序，尽管有趣的是，看看采用何种最新方式制作商业软件，有时通过查看一些经典软件问题的实现(如解析器)可以放松一下，词法分析器和抽象语法树。 如果您没有上学或未能参加编译器课程，那么至少要知道这个软件工程领域已经存在并且可以访问是非常重要的。

It's so nice to discover open source projects that I didn't know existed. One such project I just stumbled upon while doing research for a customer is "Irony," a .NET language implementation kit. From the CodePlex site:

发现我不知道存在的开源项目真是太好了。 我在为客户进行研究时偶然发现的一个这样的项目是.NET语言实现工具包“ Irony” 。 从CodePlex网站：

Irony is a development kit for implementing languages on .NET platform. Unlike most existing yacc/lex-style solutions Irony does not employ any scanner or parser code generation from grammar specifications written in a specialized meta-language. In Irony the target language grammar is coded directly in c# using operator overloading to express grammar constructs. Irony's scanner and parser modules use the grammar encoded as c# class to control the parsing process. See the expression grammar sample for an example of grammar definition in c# class, and using it in a working parser.

Irony是用于在.NET平台上实现语言的开发套件。 与大多数现有的yacc / lex风格解决方案不同，Irony不采用任何以专门的元语言编写的语法规范生成任何扫描器或解析器代码。 在Irony中，使用运算符重载来表达语法结构，直接在c＃中对目标语言语法进行编码。 Irony的扫描器和解析器模块使用编码为c＃类的语法来控制解析过程。 有关c＃类中语法定义的示例，请参见表达式语法示例，并在工作的解析器中使用它。

Irony includes "simplified grammars for C#, Scheme, SQL, GwBasic, JSON and others" to learn from. There are different kinds of parsers that are grammar generators you might be familiar with. For example, ANTLR is a what's called a LL(*) grammar generator, while Irony is a LALR (Look Ahead Left to Right) grammar generator.

具有讽刺意味的是，它包括“ C＃，Scheme，SQL，GwBasic，JSON等的简化语法”供您学习。 语法生成器是您可能熟悉的各种解析器。 例如，ANTLR是所谓的LL(*)语法生成器，而Irony是LALR(从左到右向前看)语法生成器。

Here's a very basic SQL statement for getting a show from my Podcast database:

这是一个非常基本SQL语句，用于从我的Podcast数据库中获取节目：

SELECT ID, Title FROM Shows WHERE ID = 1

Here's the Irony Parse Tree as viewed in the irony Grammar Explorer:

这是在具有讽刺意味的语法浏览器中查看的具有讽刺意味的分析树：

Typically, in my experience, when creating a parser one will use a DSL (Domain Specific Language) like the GOLD Meta Language building on the BNF (Backus-Naur Form) expression grammar. These domain specific languages are tightly optimized to express exactly how a language is structured and how it should be parsed. You learn a language to create languages.

通常，以我的经验，创建解析器时，将使用DSL(特定于域的语言)，例如在BNF(Backus-Naur Form)表达语法上建立的GOLD元语言。 这些领域特定的语言经过严格优化，以准确表达一种语言的结构以及如何对其进行解析。 您学习一种语言来创建语言。

Remember in the Irony intro text earlier? Let me repeat:

还记得先前在Irony简介文字中的内容吗？ 让我重复一遍：

Unlike most existing yacc/lex-style solutions Irony does not employ any scanner or parser code generation from grammar specifications written in a specialized meta-language. In Irony the target language grammar is coded directly in c# using operator overloading to express grammar constructs.

与大多数现有的yacc / lex风格解决方案不同，Irony不采用任何以专门的元语言编写的语法规范生成任何扫描器或解析器代码。 在Irony中，使用运算符重载来表达语法结构，直接在c＃中对目标语言语法进行编码。

What Roman from Irony has done here is use C# language constructs as if it's a DSL. A fluent parser, as it were. So he's using C# classes and methods to express the language grammar. It's a very interesting and powerful idea if you are interested in creating DSLs but not interested in learning other parsers like GOLD. Plus, it's just fun.

Irony的Roman在这里所做的就是使用C＃语言构造，就好像它是DSL。 流畅的解析器，就像以前一样。 因此，他正在使用C＃类和方法来表达语言语法。 如果您对创建DSL感兴趣但对学习其他解析器(例如GOLD)不感兴趣，那么这是一个非常有趣且功能强大的想法。 另外，这很有趣。

He has a very rich bass class called Grammar that you derive from, like:

他有一个非常丰富的低音类，称为语法，您可以从中获得，例如：

[Language("SQL", "89", "SQL 89 grammar")]
public class SqlGrammar : Grammar {
   public SqlGrammar() : base(false) { //SQL is case insensitive
   ...

But instead of a grammar language like this (simplified by me) to express a SQL SELECT Statement:

但是，不是像这样的语法语言(由我简化)来表达SQL SELECT语句：

! =============================================================================
! Select Statement
! =============================================================================
<SELECT Stm> ::= SELECT <COLUMNS> <INTO Clause> <FROM Clause> <WHERE Clause> 
<GROUP Clause> <HAVING Clause> <ORDER Clause><COLUMNS> ::= <RESTRICTION> '*' | 
<RESTRICTION> <COLUMN List>...snip for clarity...<RESTRICTION> ::= ALL | 
DISTINCT |<AGGREGATE> ::= Count '(' '*' ')' | Count '(' <EXPRESSION> ')' |
 Avg '(' <EXPRESSION> ')' | Min '(' <EXPRESSION> ')' | Max '(' <EXPRESSION> ')' |
 StDev '(' <EXPRESSION> ')' | StDevP '(' <EXPRESSION> ')' | Sum '(' <EXPRESSION> ')' |
 Var '(' <EXPRESSION> ')' | VarP '(' <EXPRESSION> ')'<INTO Clause> ::= INTO Id |
<FROM Clause> ::= FROM <ID List> <JOIN Chain><JOIN Chain> ::= <JOIN> <JOIN Chain> |
...snip for clarity... 

You'd have something like this instead, again, simplified so this doesn't turn into a giant listing of code rather than a blog post.

同样，您会得到类似的简化，因此这不会变成大量的代码清单，而不会变成博客文章。

//Select stmt
selectStmt.Rule = SELECT + selRestrOpt + selList + intoClauseOpt + fromClauseOpt + whereClauseOpt + groupClauseOpt + havingClauseOpt + orderClauseOpt;
selRestrOpt.Rule = Empty | "ALL" | "DISTINCT";
selList.Rule = columnItemList | "*";
columnItemList.Rule = MakePlusRule(columnItemList, comma, columnItem);
columnItem.Rule = columnSource + aliasOpt;aliasOpt.Rule = Empty | asOpt + Id;
asOpt.Rule = Empty | AS;columnSource.Rule = aggregate | Id;
aggregate.Rule = aggregateName + "(" + aggregateArg + ")";
aggregateArg.Rule = expression | "*"; 
aggregateName.Rule = COUNT | "Avg" | "Min" | "Max" | "StDev" | "StDevP" | "Sum" | "Var" | "VarP";
intoClauseOpt.Rule = Empty | INTO + Id;fromClauseOpt.Rule = Empty | FROM + idlist + joinChainOpt; 
joinChainOpt.Rule = Empty | joinKindOpt + JOIN + idlist + ON + Id + "=" + Id;
joinKindOpt.Rule = Empty | "INNER" | "LEFT" | "RIGHT";
whereClauseOpt.Rule = Empty | "WHERE" + expression;
groupClauseOpt.Rule = Empty | "GROUP" + BY + idlist;
havingClauseOpt.Rule = Empty | "HAVING" + expression; 
orderClauseOpt.Rule = Empty | "ORDER" + BY + orderList;

Here the variables and terms that are being use to build the grammar were defined earlier like this, as an example:

例如，这里早先定义了用于构建语法的变量和术语，例如：

var SELECT = ToTerm("SELECT"); var FROM = ToTerm("FROM");var AS = ToTerm("AS"); 

You might immediately declare, Dear Reader, that this is blasphemy!  How can C# compete with a specialized DSL like the BNF? This is a C# shaped peg being shoved into a round hold. Well, maybe, but it's interesting to point out that the SQL GOLD Grammar is 259 lines and the C# version of essentially the same thing is 247 lines. Now, I'm not pointing out line numbers to imply that this is a better way or that this is even a valid 1:1 comparison. But, it's interesting that the C# class is even close. You might have assumed it would be much much larger. I think it's close because Roman, the Irony developer, has a very well factored and specialized base class for the derived class to "lean on." Each of his sample grammars are surprisingly tight.

亲爱的读者，您可能会立即宣布这是亵渎神明！ C＃如何与BNF等专用DSL竞争？ 这是一个C＃形的钉子，被钉入圆形容器中。 好吧，也许吧，但是有趣的是指出SQL GOLD语法是259行，而本质上相同的C＃版本是247行。 现在，我并不是要指出行号来暗示这是一种更好的方法，或者甚至是有效的1：1比较。 但是，有趣的是C＃类甚至很接近。 您可能已经假定它会更大。 我认为这已经很接近了，因为Irony开发人员Roman具有一个很好的分解和专门的基类，派生类可以“依靠”。 他的每个样本语法都出奇的紧密。

For example:

例如：

• "Mini" Python - ~140 lines
  “迷你” Python-约140行
• Java - ~130 lines
  Java-〜130行
• Scheme - ~200 lines
  方案-约200条线
• JSON - 39 lines
  JSON-39行

To conclude, here's the JSON grammar generator. 

总结一下，这是JSON语法生成器。

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Irony.Parsing;

namespace Irony.Samples.Json
{
    [Language("JSON", "1.0", "JSON data format")]
    public class JsonGrammar : Grammar
    {
        public JsonGrammar()
        {
            //Terminals 
            var jstring = new StringLiteral("string", "\"");
            var jnumber = new NumberLiteral("number"); var comma = ToTerm(",");

            //Nonterminals 
            var jobject = new NonTerminal("Object");
            var jobjectBr = new NonTerminal("ObjectBr");
            var jarray = new NonTerminal("Array");
            var jarrayBr = new NonTerminal("ArrayBr");
            var jvalue = new NonTerminal("Value");
            var jprop = new NonTerminal("Property");
            //Rules 
            jvalue.Rule = jstring | jnumber | jobjectBr | jarrayBr | "true" | "false" | "null";
            jobjectBr.Rule = "{" + jobject + "}";
            jobject.Rule = MakeStarRule(jobject, comma, jprop);
            jprop.Rule = jstring + ":" + jvalue;
            jarrayBr.Rule = "[" + jarray + "]";
            jarray.Rule = MakeStarRule(jarray, comma, jvalue);
            //Set grammar root 
            this.Root = jvalue; MarkPunctuation("{", "}", "[", "]", ":", ",");
            this.MarkTransient(jvalue, jarrayBr, jobjectBr);
        }
    }
}

Pretty clever stuff, and a well put together project and solution that is well structured. I could myself using this in a C# or Compiler class to teach some of these concepts. It's also a great little tool for creating small languages of your own. Perhaps you have a Wiki-dialect that's specific to your company and you want to get rid of all that nasty manual parsing? Or many you have an old custom workflow engine or custom expression system embedded in your application and never got around to changing all your parsing to a proper grammar? Maybe now is the time to get that little language you've been thinking about off the ground!

非常聪明的东西，以及结构合理的项目和解决方案。 我自己可以在C＃或Compiler类中使用它来教授其中一些概念。 它也是创建自己的小语言的好工具。 也许您有一个特定于您公司的Wiki方言，并且想要摆脱所有讨厌的手动解析？ 还是许多人在您的应用程序中嵌入了旧的自定义工作流引擎或自定义表达系统，却从未将所有的解析更改为适当的语法？ 也许现在是时候让您开始思考的那种小语言了！

I encourage you, Dear Reader, to support open source projects like this. Why not go leave a comment today on your favorite open source project's site and just let them know you appreciate what they're doing?

亲爱的读者，我鼓励您支持这样的开源项目。 为什么不今天在您喜欢的开放源代码项目的站点上发表评论，只让他们知道您对他们的工作表示赞赏？

翻译自: https://www.hanselman.com/blog/the-weekly-source-code-59-an-open-source-treasure-irony-net-language-implementation-kit

知乎周源微信