.NET 分布式 搜索 elasticsearch.net NEST

我们公司（电商）目前使用 elasticsearch。对于一个3亿数据量索引查询速度都是在毫秒级别，不过实现都是基于JAVA的client. 本人因为是.NET出身，虽然现在做JAVA。但是对.NET情有独钟。所以闲暇时间研究了下 elasticsearch的.NET 的CLIENT的NEST.希望.NET人员多使用 elasticsearch 来做搜索服务 解决.NET一直没有好的搜索服务。本人以前也研究过 Lucene.Net。但是无法实现集群及分布式。

本人也参考了很多前辈博客和和本公司做搜索的同事 和 google。走了很多弯路。所以希望后来人别走很多弯路。

Elasticsearch一般至少最少3台服务。2台服务器时候1一台是主分片。另外一台永远都是副本所以在服务，无法实现集群式搜索。只能实现主从备份模式。

所以服务器建议3台以上。这样分片才会平衡到3台服务器上。

参考资料

https://www.nuget.org/packages/Nest

http://blog.csdn.net/huwei2003/article/details/40980929

https://www.elastic.co/guide/en/elasticsearch/client/net-api/1.x/nest.html

安装Elasticsearch

网上已经有很多资料了。所以这里不再写了

1、VS2012安装NEST

PM> Install-Package NEST -version 1.7.2

我安装的是1.7.2的客户端版本。

2、创建连接

 public  static ElasticClient GetElasticClient(string indexName)
        {
            var connectString = "http://127.0.0.1:9200";
            var nodesStr = connectString.Split('|');
            var nodes = nodesStr.Select(s => new Uri(s)).ToList();
            var connectionPool = new SniffingConnectionPool(nodes);
            var settings = new ConnectionSettings(connectionPool);
            if (!string.IsNullOrWhiteSpace(indexName))
            {
                settings.SetDefaultIndex(indexName);
            }
            var client = new ElasticClient(settings);
            return client;

        }

3、创建索引

创建索引只需要调用一次即可。

 索引结构如下：

[ElasticType(IdProperty = "id", Name = "order")]
public class Order
{
 [ElasticProperty(Name = "id", Type = FieldType.Long, Index = FieldIndexOption.NotAnalyzed)]
  public long Id { get; set; }


public string Content { get; set; }
[ElasticProperty(Name = "firstname", Type = FieldType.String, Index = FieldIndexOption.Analyzed)]
public string Firstname { get; set; }

[ElasticProperty(Name = "lastname", Type = FieldType.String, Index = FieldIndexOption.NotAnalyzed)]
public string Lastname { get; set; }


 [ElasticProperty(Name = "totalprice", Type = FieldType.Double, Index = FieldIndexOption.NotAnalyzed)]
public Double TotalPrice { get; set; }

[ElasticProperty(Name = "createtime", Type = FieldType.Date, Index = FieldIndexOption.Analyzed)]
public DateTime Createtime { get; set; }

    }

[ElasticType(IdProperty = “id”, Name = “order”)]

这个IdProperty字段属性是搜索在做分片时候需要HASH对象。所以选择这个可以根据数据库的分区键或者数据库主键来选择具体哪个字段来做IdProperty。一般都是选择数据库主键

Content：这个字段未定义属性。 所以这个字段不能拿来做查询条件。

要做查询条件的字段都必须加上

[ElasticProperty(Name = “lastname”, Type = FieldType.String, Index = FieldIndexOption.NotAnalyzed)]
或者

[ElasticProperty(Name = “createtime”, Type = FieldType.Date, Index = FieldIndexOption.Analyzed)]

这两个的区别是
NotAnalyzed 是不分词只做索引字段

Analyzed 作为查询字段并且分词

所以创建索引必须规划好每个字段是否分词及是否查询。

      /// <summary>
        /// 创建索引
        /// </summary>
        /// <param name="indexName"></param>
        private static void CreateIndex(string indexName)
        {

            var client = ElasticsearchHelper.GetElasticClient("");
            //client 不能给 default
            client.CreateIndex(r => r.Index(indexName).NumberOfReplicas(1).NumberOfShards(20));
            client.Map<Order>(m => m.MapFromAttributes()); 
        }

这里是创建ORDER的索引并且每个分片一个副本 总共20个分片。

20分片实际要根据业务数据量多少来决定。

4、删除索引

      /// <summary>
        /// 删除索引
        /// </summary>
        private static void DeleteIndex()
        {
            var client = ElasticsearchHelper.GetElasticClient("order");
            var result = client.DeleteIndex(r => r.Index("order"));
            System.Console.WriteLine(result.IsValid);
        }

5、写入数据

    private static  void WriteIndex()
        {
            var client = ElasticsearchHelper.GetElasticClient("order");
            long i = GetMax();
            while (true)
            {
                i++;
                SetMax(i);
                Order order = new Order()
                {
                    Id = i,
                    Firstname = "我爱天安门",
                    Lastname = "外派",
                    TotalPrice =Convert.ToDouble(i),
                    Content = "天使3412341234" + i.ToString(),
                    Createtime = DateTime.Now
                };
                var addResult = client.Index(order);
                System.Console.WriteLine("第" + i + "行记录" + addResult.Created);

            }

        }

        static long GetMax()
        {
            long reslt = 0;
            try
            {
                string k = System.IO.File.ReadAllText("1.txt");
                reslt = Convert.ToInt64(k);
            }
            catch (Exception)
            {

            }
            return reslt;
        }

        static long SetMax(long max)
        {
            long reslt = 0;
            try
            {
                System.IO.File.WriteAllText("1.txt", max.ToString());
            }
            catch (Exception)
            {

            }
            return reslt;
        }  

6、查询

#region 求和
var client = ElasticsearchHelper.GetElasticClient("order");
QueryContainer termQuery = new TermQuery() { Field = "lastname", Value = "求和" };
var result = client.Search<Order>(s => s
                .Aggregations(a => a
                    .Sum("my_sum_agg", sa => sa
                        .Field(p => p.TotalPrice)
                    )
                )
            );
var agg = result.Aggs.Sum("my_sum_agg");


var searchResults = client.Search<Order>(s => s
              //  .Query(termQuery)  //带筛选条件
                .Aggregations(r => r.Terms("firstname", r1 => r1.Field(r2 => r2.Lastname)
                    .OrderAscending("sumprice")
                    .Aggregations(y => y.Sum("sumprice", y1 => y1.Field(y2 => y2.TotalPrice))))));

var carTypes = searchResults.Aggs.Terms("firstname");
            List<double> re=new List<double>();
            foreach (var carType in carTypes.Items)
            {
                string key = carType.Key;
                System.Console.WriteLine("key:" + key + " total:" + carType.Sum("sumprice").Value);
                re.Add((double)carType.Sum("sumprice").Value);
            }
            //List<SumTotalPrice> orders = searchResults.Documents.ToList();
         //   System.Console.WriteLine(orders.Count() + " total:" + searchResults.Total);
            //System.Console.WriteLine(searchResults.RequestInformation);
            System.Console.ReadLine();

            #endregion


  #region 多条件精确匹配
            var client = ElasticsearchHelper.GetElasticClient("order");
            QueryContainer termQuery = new TermQuery() { Field = "firstname", Value = "测试" };
            QueryContainer wholeWordQuery = new TermQuery() { Field = "lastname", Value = "addddd" };
            //非索引字段。无法查询 此查询条件无效
            QueryContainer termQuery1 = new TermQuery() { Field = "content", Value = "  天空是百度的1696" };
            termQuery = (termQuery || wholeWordQuery) || termQuery1;

            var searchResults = client.Search<Order>(s => s
                .From(0)
                .Size(10)
                .Query(termQuery)
                );
            List<Order> orders = searchResults.Documents.ToList();
            System.Console.WriteLine(orders.Count() + " total:" + searchResults.Total);
            System.Console.ReadLine();

            #endregion

               #region 精确匹配
            var client = ElasticsearchHelper.GetElasticClient("order");
            var searchResults = client.Search<Order>(s => s
              .From(0)
              .Size(10)
              .Query(q => q.Term(r => r.OnField(k => k.Firstname).Value("棉花")))
              .Highlight(h => h.OnFields(e => e.OnField("firstname")
                                .PreTags("<b style='color:black'>")
                                .PostTags("</b>")))
              .Sort(r => r.Descending().OnField(q => q.Createtime))
              );
            List<Order> orders = searchResults.Documents.ToList();

            System.Console.WriteLine(orders.Count() + " total:" + searchResults.Total);
            System.Console.ReadLine();

            #endregion

          //  //查询索引字段 带 裤的  类似 数据库的  like '裤%'
            var client = ElasticsearchHelper.GetElasticClient("order");
            var searchResults = client.Search<Order>(s => s
          .From(0)
          .Size(10)
          .QueryString("裤*")
          );
            System.Console.WriteLine(" total:" + searchResults.Total);
            System.Console.ReadLine();


            #region ID查询

            var searchResults = client.Search<Order>(s => s
               .From(0)
               .Size(10)
               .Query(q => q.Ids(r=>r.Values(19)))
               );

            #endregion

 #region 时间范围查询
            var client = ElasticsearchHelper.GetElasticClient("order");
          //  QueryContainer termQuery = new TermQuery() { Field = "firstname", Value = "测试" };
            var format = "yyyy-MM-dd'T'HH:mm:ss.fff+08:00";
            var lowerBound = DateTime.Now.AddMinutes(-20);

            var searchResults = client.Search<Order>(s => s
               // .Query(r => termQuery && r.Range(st => st.OnField(st1 => st1.Createtime).GreaterOrEquals(lowerBound, format)))
                .Query(r => r.Range(st => st.OnField(st1 => st1.Createtime).GreaterOrEquals(lowerBound, format)))
                .SortDescending(r=>r.Createtime)
                .From(0)
                .Size(10)
                );
            List<Order> orders = searchResults.Documents.ToList();
             var k=  searchResults.RequestInformation;
            System.Console.WriteLine(orders.Count() + " total:" + searchResults.Total);
            System.Console.ReadLine();

            #endregion

7、删除数据

     private static  void DeleteRowById(long id)
        {
            var client = ElasticsearchHelper.GetElasticClient("order");

            client.Delete<Order>(r => r.Id(id));

        }