ruby on rails_在Rails中用ruby构建查询语言

ruby on rails

URL parameters are the easiest way to support filtering in a rails app. If you have a /offersendpoint which returns all offers you might be interested in, then you could filter it by any property by adding query parameters, like for example /offers?category=tech&status=active&relevance=popular and so forth. Parsing these parameters from inside a Rails controller and passing them to ActiveRecord ‘s where method is trivial and works out of the box.

URL参数是在Rails应用程序中支持过滤的最简单方法。 如果您有一个/offers端点，该端点返回您可能感兴趣的所有商品，则可以通过添加查询参数按任何属性对其进行过滤，例如/offers?category=tech&status=active&relevance=popular等。 从Rails控制器内部解析这些参数，并将它们传递给ActiveRecord的where方法很简单并且可以直接使用。

A typical controller would look something like this.

一个典型的控制器看起来像这样。

Unfortunately, this only works for exact matches. If i wanted to get all offers created within a certain date or price range, I would be out of luck.

不幸的是，这仅适用于完全匹配。 如果我想在某个日期或价格范围内创建所有报价，那我将不走运。

Simple query parameters can only get us so far. They are comprised of 2 parts, a key, indicating the property, and a value. But we need a third part as well, the operator we ought to apply to the property and the value.

简单的查询参数只能使我们走到目前。 它们由两部分组成，一个是关键字，指示属性，另一个是值。 但是我们也需要第三部分，我们应该将运算符应用于属性和值。

This article goes more in-depth about the different syntax that can be used in order to construct wiser query parameters, as well as the pros and cons of each approach. Today, we will take a look at how to implement the translation between our filtering language and the language of our database.

此文章接着更深入的关于不同的语法，可以在为了构建更明智的查询参数，以及每种方法的优点和缺点来使用。 今天，我们将看看如何实现过滤语言和数据库语言之间的转换。

For the purpose of this demonstration, I chose a syntax similar to MongoDB’s query language.

为了演示的目的，我选择了类似于MongoDB查询语言的语法。

{
  field: {
    operator:value,
    operator:value
  },
  or: [
    {
      field: {
       operator:value
      },
      field: {
       operator:value
      }
    }
  ]
}

There are a couple of things going on here, so let’s break it apart

这里发生了几件事，所以让我们分开

• A field can be filtered by multiple operators , all logically connected with AND .

  一个field可以由多个operators过滤，所有operators都与AND逻辑连接。

• Filtering by multiple fields is equivalent to AND -ing them together. You can, however, specify to use OR instead. In the absence of a logical operator, AND is assumed. There can be as many levels of nesting using combinations of AND and OR as desired.

  通过多种过滤fields相当于AND -ing在一起。 但是，您可以指定使用OR代替。 在没有逻辑运算符的情况下，假定为AND 。 可以根据需要使用AND和OR组合进行多层嵌套。

• Finally, the query in the example would be roughly translated to field operator value AND field operator value AND (field operator value OR field operator value)

  最后，示例中的查询将大致转换为field operator value AND field operator value AND (field operator value OR field operator value)

Getting this sort of structure from query parameters can be somewhat tricky. Fortunately for us, Rack can parse this out of the box without the need for a custom parameter binder, with the following format: field[operator]=value&field[operator]=value&or[[field[operator]]=value . Admittedly, it does not look pretty, especially with many levels of nesting and and or , but it is effective.

从查询参数获取这种结构可能有些棘手。 对于我们来说幸运的是，Rack可以使用以下格式，无需使用自定义参数绑定器即可立即进行解析： field[operator]=value&field[operator]=value&or[[field[operator]]=value 。 诚然，它看起来并不漂亮，特别是在嵌套and和or许多级别中，但它是有效的。

翻译 (Translation)

Assuming we have managed to parse our query string into the desired structure, we now need to parse it and convert it into a query that a database could understand.

假设我们已经成功地将查询字符串解析为所需的结构，那么现在我们需要对其进行解析并将其转换为数据库可以理解的查询。

We will do this process in two steps.

我们将分两个步骤进行此过程。

• Translate our structure into another, intermediate representation
  将我们的结构转换为另一个中间表示
• Translate this intermediate representation into a database query
  将此中间表示形式转换为数据库查询

The reason for breaking this into two steps is to give ourselves more flexibility and ease of extension over the types of databases we could support. This intermediate representation should be easier to parse than our initial structure.

将其分为两个步骤的原因是为了使自己更具灵活性，并且易于扩展我们可以支持的数据库类型。 这个中间表示应该比我们的初始结构更易于解析。

The data structure that we will use to form our intermediate representation is called an Abstract Syntax Tree(AST) . It encodes the syntax of our language using nodes for every construct we have in it. Before we proceed any further, let’s define these constructs more formally.

我们将用来形成中间表示形式的数据结构称为Abstract Syntax Tree(AST) 。 它使用节点为其中的每个构造对语言的语法进行编码。 在继续进行之前，让我们更正式地定义这些构造。

Language constructs

语言构造

The smallest construct in our filtering language is the field filter:field: {operator:value, operator:value} . It is composed of three parts, the name of the field, the operator(s) and the value(s).

在我们的过滤语言中，最小的构造是字段过滤器： field: {operator:value, operator:value} 。 它由三部分组成，字段名称，运算符和值。

Next is the filter group: field: {operator:value}, field: {operator:value} . It is a logical grouping of 1 or several field filters.

接下来是过滤器组： field: {operator:value}, field: {operator:value} 。 它是1个或几个字段过滤器的逻辑分组。

And finally, we have an expression. An expression is just several filter groups connected together with a logical operator. To help us in our parser implementation, we can write down the Backus-Naur form of our grammar, explicitly defining the recursive relations between our constructs.

最后，我们有一个表达。 表达式只是通过逻辑运算符连接在一起的几个过滤器组。 为了帮助我们执行解析器，我们可以记下语法的Backus-Naur形式，显式定义构造之间的递归关系。

• <filter>: field: {operator:value}

  <过滤器>： field: {operator:value}

• <logical operator>: and | or

  <逻辑运算符>： and | or and | or

• <filter group>: (<filter> | <expression>)*
  <过滤器组>：(<过滤器| | <表达式>)*
• <expression>: (<logical operator> <filter group>)
  <表达式>：(<逻辑运算符> <过滤器组>)

抽象语法树 (Abstract Syntax Tree)

Back to our syntax tree. Every elementary construct, one that is not built up using others, will be represented by a node. A class hierarchy could look something like this:

回到我们的语法树。 每个基本构造，一个没有使用其他构造的构造，都将由一个节点表示。 类层次结构可能看起来像这样：

All the different tokens will be represented by their types as well.

所有不同的令牌也将由它们的类型表示。

Now, having defined our grammar formally, implementing the parser is simply implementing the recursive equations:

现在，已经正式定义了语法，实现解析器就是简单地实现递归方程：

Invoking our parser withparser.parse will give us back the root node of our AST. In our case, it will be very similar to a binary tree, where the leaf nodes will be descendants ofFilterNode , and the intermediate nodes will have two children, left and right, as implemented by BinaryNode .

使用parser.parse调用解析器将使我们返回AST的根节点。 在我们的例子中，它将非常类似于二叉树，其中叶节点将成为FilterNode后代，中间节点将具有两个子节点，分别由BinaryNode实现。

Implementing the lexer object may be the more complicated part, and I may revisit this in a 2nd part, or add a link to a github gist.

实现lexer对象可能是更复杂的部分，我可能会在第二部分中再次介绍它，或者添加指向github gist的链接。

Traversing our tree

遍历我们的树

Now that we have our intermediate representation of the query, we need to translate it into a database-understandable query. To make the code cleaner and easier to extend, we will use the Visitor Pattern. In short, the Visitor pattern solves the problem of extending an existing data structure with new algorithms. In our case, the data structure is the Abstract Syntax Tree, and the different algorithms will be the translations between different database query languages. It does this by separating the implementation in two, the data structure hold all the necessary data, and the visitor object itself implements the required algorithm. It visits each node in the collection, performs a computation, stores the result in its private state, and moves on to the next one. For a detailed description of the visitor pattern and its use in implementing parsers, you can check out this amazing post, which was the inspiration for my own post.

现在我们有了查询的中间表示，我们需要将其转换为数据库可理解的查询。 为了使代码更简洁和易于扩展，我们将使用“访问者模式”。 简而言之，Visitor模式解决了使用新算法扩展现有数据结构的问题。 在我们的例子中，数据结构是抽象语法树，而不同的算法将是不同数据库查询语言之间的转换。 它通过将实现分为两部分来实现，数据结构保存所有必需的数据，而访问者对象本身实现所需的算法。 它访问集合中的每个节点，执行计算，将结果存储为私有状态，然后继续进行下一个。 有关访问者模式及其在实现解析器中的用法的详细说明，您可以查看这篇神奇的帖子，这是我自己的帖子的灵感。

Back to our visitor. For the purposes of simplicity, we will stick with supporting only SQL databases. Fortunately for us, ActiveRecord already uses a library for generating SQL strings, called Arel , which works very similarly to our own parser and AST. Arel builds up an AST when you invoke its predicate methods, and spits out the SQL with the help of the Visitor pattern. The library has visitor objects implemented for every database it supports, and it picks the correct implementation based on your configuration. Instead of reinventing the wheel, we can take advantage of this implementation, and delegate all of the SQL-generation to Arel. Our task then is to map our own AST into one that Arel can understand.

回到我们的访客。 为了简单起见，我们将坚持仅支持SQL数据库。 对我们来说幸运的是，ActiveRecord已经使用了一个用于生成SQL字符串的库，称为Arel ，该库的工作方式与我们自己的解析器和AST非常相似。 当您调用AST的谓词方法时， Arel会构建AST，并在Visitor模式的帮助下吐出SQL。 该库为它支持的每个数据库实现了访问者对象，并根据您的配置选择正确的实现。 无需重新设计轮子，我们可以利用此实现，并将所有SQL生成委派给Arel。 然后，我们的任务是将自己的AST映射到Arel可以理解的AST中。

Visitor implementations usually rely on double-dispatch, where each node accepts a visitor object, def accept(visitor) , and then passes it’s self reference to the visitor, def visit(node) . The reason for this double dispatching is because we don’t know either the visitor or the node type until runtime. The visitor would then override visit for every type of node, and rely on method overloading to invoke the correct version. Ruby, however, is a dynamically-typed language, so our approach will be slightly different.

访问者实现通常依赖于双调度，其中每个节点都接受一个访问者对象def accept(visitor) ，然后将其self引用传递给访问者def visit(node) 。 进行这种双重调度的原因是，直到运行时我们才知道访问者或节点类型。 然后，访问者将覆盖每种节点类型的visit ，并依靠方法重载来调用正确的版本。 但是，Ruby是一种动态类型的语言，因此我们的方法会略有不同。

First, we define the Visitor class:

首先，我们定义Visitor类：

We use ruby’s dynamic method invocation feature, using our node’s class as a descriminator. For example, when visit is invoked with a AndNode , it will delegate to the correct visit_AndNode method.

我们使用ruby的动态方法调用功能，将节点的类用作描述符。 例如，当visit调用与AndNode ，它会委托给正确的visit_AndNode方法。

The collector object will hold the result as we build it up by visiting nodes.

当我们通过访问节点建立结果时，收集器对象将保存结果。

Here we are completely relying on Arel’s methods for implementing our own node types, which means that as long as we don’t have any operators that Arel itself does not support, we will have no problems.

在这里，我们完全依靠Arel的方法来实现自己的节点类型，这意味着只要我们没有Arel本身不支持的任何运算符，我们就不会有问题。

One note of caution when using this approach: we must make sure the values we receive via URL parameters are properly sanitized. I chose to do this inside the visitor , because it gives the most control over the type of sanitization (for example, sanitizing a LIKE query would require using sanitize_sql_like instead of sanitize_sql ).

使用这种方法时需要注意的一点是：我们必须确保正确清理了通过URL参数收到的值。 我选择在visitor内执行此操作，因为它可以最大程度地控制清理类型(例如，清理LIKE查询将需要使用sanitize_sql_like而不是sanitize_sql )。

To wrap things up, using all parts to get to a simple query like in our starting example:

总结一下，使用所有部分来进行简单的查询，例如在我们的示例中：

parser = Parser.new(Lexer.new(query_params))
visitor = ArelVisitor.new(Offer.arel_table)
root_node = parser.parse
root_node.accept(visitor)Offer.where(visitor.to_query)

References:RESTful filtering: https://www.moesif.com/blog/technical/api-design/REST-API-Design-Filtering-Sorting-and-Pagination/

参考：RESTful过滤： https ： //www.moesif.com/blog/technical/api-design/REST-API-Design-Filtering-Sorting-and-Pagination/

Explanation of AST/Visitor/Parser: https://ruslanspivak.com/lsbasi-part7/

AST / Visitor / Parser的说明： https : //ruslanspivak.com/lsbasi-part7/

Using Arel to compose queries: http://radar.oreilly.com/2014/05/more-than-enough-arel.html

使用Arel编写查询： http ://radar.oreilly.com/2014/05/more-than-enough-arel.html

翻译自: https://medium.com/web-factory-llc/building-a-query-language-in-ruby-on-rails-54a597dc7def

ruby on rails