Elastic Search NEST 研究（四）--NEST客户端在WebApi项目中的使用

NEST客户端在WebApi项目中的使用

本项目的源码已发布在https://gitee.com/lucyliang01/esapi.git

NEST是elastic search为.net 提供的高级客户端依赖组件。

这里我们会建一个web api2项目，进行演示在.net中使用NEST实现文档的增删改查和全文检索

创建web api项目

创建web api2项目，并且修改属性目标框架为.net framework 4.6.1

在nuget中找到NEST依赖，并且安装目前版本7.10.1

创建 ESHelper帮助类文件

配置链接

这里因为对es的连接设置成private static只在创建的时候初始化一次

//单机连接的方式，默认使用articles索引
private static ConnectionSettings settings = new ConnectionSettings(new Uri("http://localhost:9200")).DefaultIndex("articles");
//创建client
private static ElasticClient client = new ElasticClient(settings);

使用attribute自动映射

根据映射的字段创建类Article,并且使用attribute规定映射规则。

1）Number 表示字段类型为数字

2）Text 表示字段类型为字符串，可以进行索引

​ Analyzer 可以规定索引时的分词工具

​ Index 为true表示建立索引，并且可以被检索

3）Keyword 表示字段类型为字符串，但是不可以进行索引

4）Date 表示字段类型为日期

[ElasticsearchType(RelationName = "articles")]
    public class Article
    {
        /// <summary>
        /// id
        /// </summary>
        [Number]
        public long Id { get; set; }
        /// <summary>
        /// 标题
        /// </summary>
        [Text(Analyzer = "ik_max_word", Index = true)]
        public string Title { get; set; }
        /// <summary>
        /// 类型 新闻还是招聘
        /// </summary>
        [Keyword]
        public string Type { get; set; }
        /// <summary>
        /// 内容
        /// </summary>
        [Text(Analyzer = "ik_max_word", Index = true)]
        public string Content { get; set; }
        /// <summary>
        /// 新闻作者
        /// </summary>
        [Keyword]
        public string Author { get; set; }
        /// <summary>
        /// 招聘公司
        /// </summary>
        [Text(Analyzer = "ik_max_word", Index = true)]
        public string Company { get; set; }
        /// <summary>
        /// 招聘公司地址
        /// </summary>
        [Text(Analyzer = "ik_max_word", Index = true)]
        public string CompanyAddress { get; set; }
        /// <summary>
        /// 创建时间
        /// </summary>
        [Date]
        public DateTime CreateTime { get; set; }
        /// <summary>
        /// 访问路径
        /// </summary>
        [Keyword]
        public string WebPath { get; set; }
    }

创建映射

在ESHelper中创建映射的方法

/// <summary>
        /// 创建映射
        /// </summary>
        /// <returns></returns>
        public static bool CreateMapping()
        {
            try
            {
                //映射
                CreateIndexResponse createIndexResponse = client.Indices.Create("articles", c => c.Map<Article>(m => m.AutoMap()));
                return true;
            }
            catch (Exception ex)
            {
                return false;
            }
        }

在ESController中创建webapi对创建索引的请求

 /// <summary>
        /// 创建映射
        /// </summary>
        /// <returns></returns>
        [HttpPost, Route("mapping")]
        public IHttpActionResult Mapping()
        {
            return Ok(ESHelper.CreateMapping());

        }

创建或者修改单个文档

在ESHelper中创建文档，并且在ESController中创建添加文档的接口，使用postman进行测试

        /// <summary>
        /// 创建单个文档
        /// </summary>
        /// <param name="client"></param>
        /// <param name="article"></param>
        /// <returns></returns>
        public static dynamic CreateArticle(Article article)
        {
            var indexResponse = client.IndexDocument<Article>(article);
            return new { flag = true, msg = "操作成功" };
        }

  /// <summary>
        /// 添加一个文档
        /// </summary>
        /// <param name="article"></param>
        /// <returns></returns>
        [HttpPost, Route("add")]
        public IHttpActionResult Add(Article article)
        {
            return Ok(ESHelper.CreateArticle(article));

        }

批量创建文档

在ESHelper中批量创建文档，并且在ESController中创建批量添加文档的接口，使用postman进行测试

 /// <summary>
        /// 批量新增
        /// </summary>
        /// <param name="client"></param>
        /// <param name="list"></param>
        /// <returns></returns>
        public static dynamic CreateBulk(List<Article> list)
        {

            var bulkAllObservable = client.BulkAll(list, b => b
                                .Index("articles")
                                .BackOffTime("30s")
                                .BackOffRetries(2)
                                .RefreshOnCompleted()
                                .MaxDegreeOfParallelism(Environment.ProcessorCount)
                                .Size(1000)
                                    )
                                .Wait(TimeSpan.FromMinutes(15), next =>
                                {
                                    // do something e.g. write number of pages to console
                                });
            return new { flag = true, msg = "操作成功" };
        }

 /// <summary>
        /// 批量添加文档
        /// </summary>
        /// <param name="list"></param>
        /// <returns></returns>

        [HttpPost, Route("addBulk")]
        public IHttpActionResult addBulk(List<Article> list)
        {
            return Ok(ESHelper.CreateBulk(list));

        }

批量删除

批量删除我们是先根据ids查询出对应的文档，然后进行批量删除

• 根据ids查询多个记录

  在ESHelper中根据id批量获取文档

    /// <summary>
          /// 根据id查询
          /// </summary>
          /// <param name="ids"></param>
          /// <returns></returns>
          public static List<Article> GetByIds(List<long> ids)
          {
              var searchResponse = client.Search<Article>(s => s.Query(q => q.Ids(m => m.Values(ids))));
              return searchResponse.Documents.ToList();
  
          }
  

• 批量删除

  在ESHelper中批量删除，并且在ESController中创建批量删除文档的接口，使用postman进行测试

    /// <summary>
          /// 批量删除
          /// </summary>
          /// <param name="client"></param>
          /// <param name="list"></param>
          /// <returns></returns>
          public static dynamic DeleteBulk(List<Article> list)
          {
              var bulkResponse = client.DeleteMany(list);
              return new { flag = true, msg = "操作成功" };
          }
  

   /// <summary>
          /// 批量删除
          /// </summary>
          /// <param name="ids"></param>
          /// <returns></returns>
          [HttpPost, Route("delete")]
          public IHttpActionResult Delete(List<long> ids)
          {
              //根据id获取list
              var list = ESHelper.GetByIds(ids);
              return Ok(ESHelper.DeleteBulk(list));
  
          }
  

  

全文检索

项目中的全文检索解决方案就是multi match +highlight

multi match 可以对多个字段进行检索并且通过minimum_should_match和boost提高匹配度，

最终的显示结果使用highlight高亮。

在ESHelper中创建全文检索的方法，并且在ESController中创建搜索文档的接口，使用postman进行测试

 /// <summary>
        /// 全文检索
        /// </summary>
        /// <param name="page"></param>
        /// <param name="keyword"></param>
        /// <returns></returns>
        public static Dictionary<string, object> Search(int page, string keyword)
        {
            try
            {
                int size = 10;
                int from = (page - 1) * size;
                var searchResponse = client.Search<Article>(s => s
                                          .From(from)
                                          .Size(size)
                                          .Query(q =>
                                          q.MultiMatch(c => c
                                                .Fields(f => f.Field(a => a.Title, 10).Field(a => a.Content).Field(a => a.Company, 10))
                                                .Operator(Operator.Or)//只要有一个词在文档中出现都可以
                                                .MinimumShouldMatch(new MinimumShouldMatch("50%"))
                                                .Query(keyword)
                                            ))
                                             .Highlight(h => h
                                            .PreTags("<span style='color:red;'>")
                                            .PostTags("</span>")
                                            .FragmentSize(100)
                                            .NoMatchSize(150)
                                            .Fields(
                                                fs => fs
                                                    .Field(p => p.Title),
                                                 fs => fs
                                                    .Field(p => p.Company),
                                                fs => fs
                                                     .Field(p => p.Content)
                                            )
                                          )
                                      );

                var hits = searchResponse.HitsMetadata.Hits;
                var total = searchResponse.Total;

                foreach (var hit in hits)
                {
                    foreach (var highlightField in hit.Highlight)
                    {
                        if (highlightField.Key == "title")
                        {
                            foreach (var highlight in highlightField.Value)
                            {
                                hit.Source.Title = highlight.ToString();
                            }
                        }

                        if (highlightField.Key == "content")
                        {
                            foreach (var highlight in highlightField.Value)
                            {
                                hit.Source.Content = highlight.ToString();
                            }
                        }

                        if (highlightField.Key == "company")
                        {
                            foreach (var highlight in highlightField.Value)
                            {
                                hit.Source.Company = highlight.ToString();
                            }
                        }
                    }
                }
                var hitsJson = Newtonsoft.Json.JsonConvert.SerializeObject(hits);

                List<Article> list = new List<Article>();
                foreach (var item in hits)
                {
                    list.Add(item.Source);
                }
                Dictionary<string, object> result = new Dictionary<string, object>();
                result.Add("list", list);
                result.Add("total", total);

                return result;

            }
            catch (Exception ex)
            {

                throw;
            }
        }

 /// <summary>
        /// 全文检索
        /// </summary>
        /// <param name="page"></param>
        /// <param name="keyword"></param>
        /// <returns></returns>
        [HttpGet, Route("search")]
        public IHttpActionResult Search(int? page,string keyword)
        {
           int pageIndex = page ?? 1;
            return Ok(ESHelper.Search(pageIndex, keyword));

        }

ESHelper.cs代码

 public class ESHelper
    {
        private static ConnectionSettings settings = new ConnectionSettings(new Uri("http://localhost:9200")).DefaultIndex("articles");

        private static ElasticClient client = new ElasticClient(settings);

        /// <summary>
        /// 获取client
        /// </summary>
        /// <returns></returns>
        public static ElasticClient GetClient()
        {
            return client;
        }
        /// <summary>
        /// 创建映射
        /// </summary>
        /// <returns></returns>
        public static bool CreateMapping()
        {
            try
            {
                //映射
                CreateIndexResponse createIndexResponse = client.Indices.Create("articles", c => c.Map<Article>(m => m.AutoMap()));
                return true;
            }
            catch (Exception ex)
            {
                return false;
            }
        }

        /// <summary>
        /// 创建单个文档
        /// </summary>
        /// <param name="client"></param>
        /// <param name="article"></param>
        /// <returns></returns>
        public static dynamic CreateArticle(Article article)
        {
            var indexResponse = client.IndexDocument<Article>(article);
            return new { flag = true, msg = "操作成功" };
        }

        /// <summary>
        /// 批量新增
        /// </summary>
        /// <param name="client"></param>
        /// <param name="list"></param>
        /// <returns></returns>
        public static dynamic CreateBulk(List<Article> list)
        {

            var bulkAllObservable = client.BulkAll(list, b => b
                                .Index("articles")
                                .BackOffTime("30s")
                                .BackOffRetries(2)
                                .RefreshOnCompleted()
                                .MaxDegreeOfParallelism(Environment.ProcessorCount)
                                .Size(1000)
                                    )
                                .Wait(TimeSpan.FromMinutes(15), next =>
                                {
                                    // do something e.g. write number of pages to console
                                });
            return new { flag = true, msg = "操作成功" };
        }
        /// <summary>
        /// 根据id查询
        /// </summary>
        /// <param name="ids"></param>
        /// <returns></returns>
        public static List<Article> GetByIds(List<long> ids)
        {
            var searchResponse = client.Search<Article>(s => s.Query(q => q.Ids(m => m.Values(ids))));
            return searchResponse.Documents.ToList();

        }

        /// <summary>
        /// 批量删除
        /// </summary>
        /// <param name="client"></param>
        /// <param name="list"></param>
        /// <returns></returns>
        public static dynamic DeleteBulk(List<Article> list)
        {
            var bulkResponse = client.DeleteMany(list);
            return new { flag = true, msg = "操作成功" };
        }
        /// <summary>
        /// 全文检索
        /// </summary>
        /// <param name="page"></param>
        /// <param name="keyword"></param>
        /// <returns></returns>
        public static Dictionary<string, object> Search(int page, string keyword)
        {
            try
            {
                int size = 10;
                int from = (page - 1) * size;
                var searchResponse = client.Search<Article>(s => s
                                          .From(from)
                                          .Size(size)
                                          .Query(q =>
                                          q.MultiMatch(c => c
                                                .Fields(f => f.Field(a => a.Title, 10).Field(a => a.Content).Field(a => a.Company, 10))
                                                .Operator(Operator.Or)//只要有一个词在文档中出现都可以
                                                .MinimumShouldMatch(new MinimumShouldMatch("50%"))
                                                .Query(keyword)
                                            ))
                                             .Highlight(h => h
                                            .PreTags("<span style='color:red;'>")
                                            .PostTags("</span>")
                                            .FragmentSize(100)
                                            .NoMatchSize(150)
                                            .Fields(
                                                fs => fs
                                                    .Field(p => p.Title),
                                                 fs => fs
                                                    .Field(p => p.Company),
                                                fs => fs
                                                     .Field(p => p.Content)
                                            )
                                          )
                                      );

                var hits = searchResponse.HitsMetadata.Hits;
                var total = searchResponse.Total;

                foreach (var hit in hits)
                {
                    foreach (var highlightField in hit.Highlight)
                    {
                        if (highlightField.Key == "title")
                        {
                            foreach (var highlight in highlightField.Value)
                            {
                                hit.Source.Title = highlight.ToString();
                            }
                        }

                        if (highlightField.Key == "content")
                        {
                            foreach (var highlight in highlightField.Value)
                            {
                                hit.Source.Content = highlight.ToString();
                            }
                        }

                        if (highlightField.Key == "company")
                        {
                            foreach (var highlight in highlightField.Value)
                            {
                                hit.Source.Company = highlight.ToString();
                            }
                        }
                    }
                }
                var hitsJson = Newtonsoft.Json.JsonConvert.SerializeObject(hits);

                List<Article> list = new List<Article>();
                foreach (var item in hits)
                {
                    list.Add(item.Source);
                }
                Dictionary<string, object> result = new Dictionary<string, object>();
                result.Add("list", list);
                result.Add("total", total);

                return result;

            }
            catch (Exception ex)
            {

                throw;
            }
        }
    }

    [ElasticsearchType(RelationName = "articles")]
    public class Article
    {
        /// <summary>
        /// id
        /// </summary>
        [Number]
        public long Id { get; set; }
        /// <summary>
        /// 标题
        /// </summary>
        [Text(Analyzer = "ik_max_word", Index = true)]
        public string Title { get; set; }
        /// <summary>
        /// 类型 新闻还是招聘
        /// </summary>
        [Keyword]
        public string Type { get; set; }
        /// <summary>
        /// 内容
        /// </summary>
        [Text(Analyzer = "ik_max_word", Index = true)]
        public string Content { get; set; }
        /// <summary>
        /// 新闻作者
        /// </summary>
        [Keyword]
        public string Author { get; set; }
        /// <summary>
        /// 招聘公司
        /// </summary>
        [Text(Analyzer = "ik_max_word", Index = true)]
        public string Company { get; set; }
        /// <summary>
        /// 招聘公司地址
        /// </summary>
        [Text(Analyzer = "ik_max_word", Index = true)]
        public string CompanyAddress { get; set; }
        /// <summary>
        /// 创建时间
        /// </summary>
        [Date]
        public DateTime CreateTime { get; set; }
        /// <summary>
        /// 访问路径
        /// </summary>
        [Keyword]
        public string WebPath { get; set; }
    }

ESController.cs代码

 [RoutePrefix("api/es")]
    public class ESController : ApiController
    {
        /// <summary>
        /// 创建映射
        /// </summary>
        /// <returns></returns>
        [HttpPost, Route("mapping")]
        public IHttpActionResult Mapping()
        {
            return Ok(ESHelper.CreateMapping());

        }

        /// <summary>
        /// 添加一个文档
        /// </summary>
        /// <param name="article"></param>
        /// <returns></returns>
        [HttpPost, Route("add")]
        public IHttpActionResult Add(Article article)
        {
            return Ok(ESHelper.CreateArticle(article));

        }
        /// <summary>
        /// 批量添加文档
        /// </summary>
        /// <param name="list"></param>
        /// <returns></returns>

        [HttpPost, Route("addBulk")]
        public IHttpActionResult addBulk(List<Article> list)
        {
            return Ok(ESHelper.CreateBulk(list));

        }

        /// <summary>
        /// 批量删除
        /// </summary>
        /// <param name="ids"></param>
        /// <returns></returns>
        [HttpPost, Route("delete")]
        public IHttpActionResult Delete(List<long> ids)
        {
            //根据id获取list
            var list = ESHelper.GetByIds(ids);
            return Ok(ESHelper.DeleteBulk(list));

        }

        /// <summary>
        /// 全文检索
        /// </summary>
        /// <param name="page"></param>
        /// <param name="keyword"></param>
        /// <returns></returns>
        [HttpGet, Route("search")]
        public IHttpActionResult Search(int? page,string keyword)
        {
           int pageIndex = page ?? 1;
            return Ok(ESHelper.Search(pageIndex, keyword));

        }
    }