elasticsearch-入门(一)

相关文档

esapi:https://es.xiaoleilu.com/010_Intro/10_Installing_ES.html     https://esdoc.bbossgroups.com/#/pinyin-ik

es中文社区:https://elasticsearch.cn/question/6899

比较全的api http://cwiki.apachecn.org/pages/viewpage.action?pageId=4260605  

什么是elasticsearch

elasticsearch是java基础apache lucene的开源搜索引擎，开箱即用，elasticsearch有很多默认的合理的配置

elasticsearch作用

全文检索，分布式实时文件存储,分布式实时分析引擎,可以扩展到上百台服务器处理PB级结构化数据或非结构化数据

elasticsearch安装

官网下载:https://www.elastic.co/downloads/elasticsearch解压

linux运行:./bin/elasticsearch

访问:http://localhost:9200/?pretty

表示安装成功

ES与传统数据的区别

关系型数据库:databases->tables->rows->columns

ES:indices->types->documents->Fileds

index可以理解成数据库  type可以理解为表  doument可以理解为行  filed可以理解为列

索引名词=数据库  索引动词可以理解为数据库的insert语句  索引文档如果存在 旧的将覆盖新的  ES为字段建立的索引为倒排索引 

索引CRUD

查看索引
curl localhost:9200/_cat/indices?v

删除索引,通配符形式
curl -XDELETE localhost:9200/索引*

索引起别名
curl -XPUT localhost:9200/索引/_alias/别名

查看别名
curl -XPUT localhost:9200/_cat/aliases?v

建立索引

现在我们建立1个产品的索引(insert)

posman put请求

http://127.0.0.1:9200/opcmdev/product/1

body:

{
    "productName": "法瑞思 天然椰棕床垫棕垫硬床垫定做 薄乳胶棕榈榻榻米床垫定制折叠1.2/1.5/1.8",
    "price": 10,
    "remark": "不错的床垫",
    "tags": ["家具", "床垫", "棉花"]
}

 

我们可以理解为在opcmdev数据库 product表 插入了一条产品信息  方便后面测试 多添加几条数据

http://127.0.0.1:9200/opcmdev/product/2

{
    "productName": "南极人（Nanjiren）床垫家纺 加厚立体床垫羽丝绒床垫床褥子学生宿舍",
    "price": 109,
    "remark": "不错的床垫",
    "tags": ["家具", "床垫", "棉花被"]
}

http://127.0.0.1:9200/opcmdev/product/3

{
    "productName": "￥99.00志高（CHIGO）电火锅家用多功能电炒锅电煮锅不粘多用途锅电热锅 ZG-RH2",
    "price": 99,
    "remark": "平底锅",
    "tags": ["电器", "平底锅"]
}

简单的检索文档(查询)

根据id查询

get请求:http://127.0.0.1:9200/opcmdev/product/3

不带条件的简单搜索 

get请求:http://127.0.0.1:9200/opcmdev/product/_search 默认返回前10条

{
    "took": 1,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": 3,
        "max_score": 1,
        "hits": [
            {
                "_index": "opcmdev",
                "_type": "product",
                "_id": "2",
                "_score": 1,
                "_source": {
                    "productName": "南极人（Nanjiren）床垫家纺 加厚立体床垫羽丝绒床垫床褥子学生宿舍",
                    "price": "109",
                    "remark": "不错的床垫",
                    "tags": [
                        "家具",
                        "床垫",
                        "棉花被"
                    ]
                }
            },
            {
                "_index": "opcmdev",
                "_type": "product",
                "_id": "1",
                "_score": 1,
                "_source": {
                    "productName": "法瑞思 天然椰棕床垫棕垫硬床垫定做 薄乳胶棕榈榻榻米床垫定制折叠1.2/1.5/1.8",
                    "price": "10",
                    "remark": "不错的床垫",
                    "tags": [
                        "家具",
                        "床垫",
                        "棉花被"
                    ]
                }
            },
            {
                "_index": "opcmdev",
                "_type": "product",
                "_id": "3",
                "_score": 1,
                "_source": {
                    "productName": "￥99.00志高（CHIGO）电火锅家用多功能电炒锅电煮锅不粘多用途锅电热锅 ZG-RH2",
                    "price": "99",
                    "remark": "平底锅",
                    "tags": [
                        "电器",
                        "平底锅"
                    ]
                }
            }
        ]
    }
}

通过url?q模糊搜索

http://127.0.0.1:9200/opcmdev/product/_search?q=productName:锅

DSL查询语句 

查询条件分词查询

 doman spcific language 能够完整更强大的查询功能

默认情况下会将相识度最高的 排在前面 比如搜索平底锅  那么平底锅会排在前面 而其他锅则排在后面

get请求:http://127.0.0.1:9200/opcmdev/product/_search

body:

{
    "query":{
        "match":{
            "productName":"丝绒床垫"
        }
    }
}

结果

{
    "took": 1,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": 2,
        "max_score": 1.5142827,
        "hits": [
            {
                "_index": "opcmdev",
                "_type": "product",
                "_id": "2",
                "_score": 1.5142827,
                "_source": {
                    "productName": "南极人（Nanjiren）床垫家纺 加厚立体床垫羽丝绒床垫床褥子学生宿舍",
                    "price": 109,
                    "remark": "不错的床垫",
                    "tags": [
                        "家具",
                        "床垫",
                        "棉花被"
                    ]
                }
            },
            {
                "_index": "opcmdev",
                "_type": "product",
                "_id": "1",
                "_score": 0.9389185,
                "_source": {
                    "productName": "法瑞思 天然椰棕床垫棕垫硬床垫定做 薄乳胶棕榈榻榻米床垫定制折叠1.2/1.5/1.8",
                    "price": 10,
                    "remark": "不错的床垫",
                    "tags": [
                        "家具",
                        "床垫",
                        "棉花"
                    ]
                }
            }
        ]
    }
}

查询条件不分词查询

只需要将条件修改为

{
    "query":{
        "match_phrase":{
            "productName":"丝绒床垫"
        }
    }
}

结果

{
    "took": 7,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": 1,
        "max_score": 1.1507283,
        "hits": [
            {
                "_index": "opcmdev",
                "_type": "product",
                "_id": "2",
                "_score": 1.1507283,
                "_source": {
                    "productName": "南极人（Nanjiren）床垫家纺 加厚立体床垫羽丝绒床垫床褥子学生宿舍",
                    "price": 109,
                    "remark": "不错的床垫",
                    "tags": [
                        "家具",
                        "床垫",
                        "棉花被"
                    ]
                }
            }
        ]
    }
}

范围查询

需求:查询价格大于100的产品

 get请求:http://127.0.0.1:9200/opcmdev/product/_search

5.*之前版本

{
    "query" : {
        "filtered" : {
            "filter" : {
                "range" : {
                    "price" : { "gt" : 30 } 
                }
            },
            "query" : {
                "match" : {
                    "productName" : "锅" 
                }
            }
        }
    }
}

5.*之后版本

{
    "query" : {
        "bool" : {
            "filter" : {
                "range" : {
                    "price" : { "gt" : 100 }
                }
            },
            "must" : {
                "match" : {
                    "productName" : "锅"
                }
            }
        }
    }
}

结果

 为什么结果不对呢

请求：http://127.0.0.1:9200/opcmdev/product/_mapping?pretty

结果：

{
    "opcmdev": {
        "mappings": {
            "product": {
                "properties": {
                    "price": {
                        "type": "text",
                        "fields": {
                            "keyword": {
                                "type": "keyword",
                                "ignore_above": 256
                            }
                        }
                    },
                    "productName": {
                        "type": "text",
                        "fields": {
                            "keyword": {
                                "type": "keyword",
                                "ignore_above": 256
                            }
                        }
                    },
                    "proudctName": {
                        "type": "text",
                        "fields": {
                            "keyword": {
                                "type": "keyword",
                                "ignore_above": 256
                            }
                        }
                    },
                    "remark": {
                        "type": "text",
                        "fields": {
                            "keyword": {
                                "type": "keyword",
                                "ignore_above": 256
                            }
                        }
                    },
                    "tags": {
                        "type": "text",
                        "fields": {
                            "keyword": {
                                "type": "keyword",
                                "ignore_above": 256
                            }
                        }
                    }
                }
            }
        }
    }
}

因为创建索引没有指定mapping(数据类型)默认是text导致无效  但是又不能修改数据类型参考

mapping创建参考:

https://blog.csdn.net/napoay/article/details/52012249

如果正式遇到这种情况 需要修改类型 参考

http://www.cnblogs.com/Creator/p/3722408.html

高亮搜索

get请求:http://127.0.0.1:9200/opcmdev/product/_search

参数

{
    "query": {
        "match_phrase": {
            "productName": "床垫"
        }
    },
    "highlight": {
        "fields": {
            "productName": {}
        }
    }
}

结果

{
    "took": 19,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": 2,
        "max_score": 0.90414375,
        "hits": [
            {
                "_index": "opcmdev",
                "_type": "product",
                "_id": "2",
                "_score": 0.90414375,
                "_source": {
                    "productName": "南极人（Nanjiren）床垫家纺 加厚立体床垫羽丝绒床垫床褥子学生宿舍",
                    "price": 109,
                    "remark": "不错的床垫",
                    "tags": [
                        "家具",
                        "床垫",
                        "棉花被"
                    ]
                },
                "highlight": {
                    "productName": [
                        "南极人（Nanjiren）<em>床</em><em>垫</em>家纺 加厚立体<em>床</em><em>垫</em>羽丝绒<em>床</em><em>垫</em>床褥子学生宿舍"
                    ]
                }
            },
            {
                "_index": "opcmdev",
                "_type": "product",
                "_id": "1",
                "_score": 0.90414375,
                "_source": {
                    "productName": "法瑞思 天然椰棕床垫棕垫硬床垫定做 薄乳胶棕榈榻榻米床垫定制折叠1.2/1.5/1.8",
                    "price": 10,
                    "remark": "不错的床垫",
                    "tags": [
                        "家具",
                        "床垫",
                        "棉花"
                    ]
                },
                "highlight": {
                    "productName": [
                        "法瑞思 天然椰棕<em>床</em><em>垫</em>棕垫硬<em>床</em><em>垫</em>定做 薄乳胶棕榈榻榻米<em>床</em><em>垫</em>定制折叠1.2/1.5/1.8"
                    ]
                }
            }
        ]
    }
}

前端显示我们给em这是红色 就行了

分析

aggregations聚合

类似sql的goup by

5.0之前版本

{
    "aggs":{
        "all_tags":{
            "terms":{
                "field":"tags"
            }
        }
    }
}

5.0之后版本会报错

Fielddata is disabled on text fields by default. Set fielddata=true 

因为5.x以后聚合这些操作用单独的数据结构(fielddata)缓存到内存里了 会占用大量堆空间 对性能有影响 默认没有开启

put请求:http://127.0.0.1:9200/opcmdev/_mapping/product/?pretty

结果:

{
    "acknowledged": true
}

然后我们就可以得到聚合结果

{
    "took": 44,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": 3,
        "max_score": 1,
        "hits": [
            {
                "_index": "opcmdev",
                "_type": "product",
                "_id": "2",
                "_score": 1,
                "_source": {
                    "productName": "南极人（Nanjiren）床垫家纺 加厚立体床垫羽丝绒床垫床褥子学生宿舍",
                    "price": 109,
                    "remark": "不错的床垫",
                    "tags": [
                        "家具",
                        "床垫",
                        "棉花被"
                    ]
                }
            },
            {
                "_index": "opcmdev",
                "_type": "product",
                "_id": "1",
                "_score": 1,
                "_source": {
                    "productName": "法瑞思 天然椰棕床垫棕垫硬床垫定做 薄乳胶棕榈榻榻米床垫定制折叠1.2/1.5/1.8",
                    "price": 10,
                    "remark": "不错的床垫",
                    "tags": [
                        "家具",
                        "床垫",
                        "棉花"
                    ]
                }
            },
            {
                "_index": "opcmdev",
                "_type": "product",
                "_id": "3",
                "_score": 1,
                "_source": {
                    "productName": "￥99.00志高（CHIGO）电火锅家用多功能电炒锅电煮锅不粘多用途锅电热锅 ZG-RH2",
                    "price": 99,
                    "remark": "平底锅",
                    "tags": [
                        "电器",
                        "平底锅"
                    ]
                }
            }
        ]
    },
    "aggregations": {
        "all_tags": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 2,
            "buckets": [
                {
                    "key": "具",
                    "doc_count": 2
                },
                {
                    "key": "垫",
                    "doc_count": 2
                },
                {
                    "key": "家",
                    "doc_count": 2
                },
                {
                    "key": "床",
                    "doc_count": 2
                },
                {
                    "key": "棉",
                    "doc_count": 2
                },
                {
                    "key": "花",
                    "doc_count": 2
                },
                {
                    "key": "器",
                    "doc_count": 1
                },
                {
                    "key": "平",
                    "doc_count": 1
                },
                {
                    "key": "底",
                    "doc_count": 1
                },
                {
                    "key": "电",
                    "doc_count": 1
                }
            ]
        }
    }
}

可以看到  聚合的字段被分词了 不需要分词需要给参数加上keyword

{
    "aggs":{
        "all_tags":{
            "terms":{
                "field":"tags.keyword"
            }
        }
    }
}

结果

{
    "took": 10,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": 3,
        "max_score": 1,
        "hits": [
            {
                "_index": "opcmdev",
                "_type": "product",
                "_id": "2",
                "_score": 1,
                "_source": {
                    "productName": "南极人（Nanjiren）床垫家纺 加厚立体床垫羽丝绒床垫床褥子学生宿舍",
                    "price": 109,
                    "remark": "不错的床垫",
                    "tags": [
                        "家具",
                        "床垫",
                        "棉花被"
                    ]
                }
            },
            {
                "_index": "opcmdev",
                "_type": "product",
                "_id": "1",
                "_score": 1,
                "_source": {
                    "productName": "法瑞思 天然椰棕床垫棕垫硬床垫定做 薄乳胶棕榈榻榻米床垫定制折叠1.2/1.5/1.8",
                    "price": 10,
                    "remark": "不错的床垫",
                    "tags": [
                        "家具",
                        "床垫",
                        "棉花"
                    ]
                }
            },
            {
                "_index": "opcmdev",
                "_type": "product",
                "_id": "3",
                "_score": 1,
                "_source": {
                    "productName": "￥99.00志高（CHIGO）电火锅家用多功能电炒锅电煮锅不粘多用途锅电热锅 ZG-RH2",
                    "price": 99,
                    "remark": "平底锅",
                    "tags": [
                        "电器",
                        "平底锅"
                    ]
                }
            }
        ]
    },
    "aggregations": {
        "all_tags": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
                {
                    "key": "家具",
                    "doc_count": 2
                },
                {
                    "key": "床垫",
                    "doc_count": 2
                },
                {
                    "key": "平底锅",
                    "doc_count": 1
                },
                {
                    "key": "棉花",
                    "doc_count": 1
                },
                {
                    "key": "棉花被",
                    "doc_count": 1
                },
                {
                    "key": "电器",
                    "doc_count": 1
                }
            ]
        }
    }
}

如果聚合前需要查询条件 则改参数

{
    "query":{
      "match":{
          "productName":"平底锅"
      }    
    },
    "aggs":{
        "all_tags":{
            "terms":{
                "field":"tags.keyword"
            }
        }
    }
}

结果:

{
    "took": 1,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": 1,
        "max_score": 0.51040375,
        "hits": [
            {
                "_index": "opcmdev",
                "_type": "product",
                "_id": "3",
                "_score": 0.51040375,
                "_source": {
                    "productName": "￥99.00志高（CHIGO）电火锅家用多功能电炒锅电煮锅不粘多用途锅电热锅 ZG-RH2",
                    "price": 99,
                    "remark": "平底锅",
                    "tags": [
                        "电器",
                        "平底锅"
                    ]
                }
            }
        ]
    },
    "aggregations": {
        "all_tags": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
                {
                    "key": "平底锅",
                    "doc_count": 1
                },
                {
                    "key": "电器",
                    "doc_count": 1
                }
            ]
        }
    }
}

分级汇总

比如再进行二次统计 上面结果每个tag的平均价格  因为price是text类型 就不演示结果了

{
    "aggs":{
        "all_tags":{
            "terms":{
                "field":"tags.keyword"
            },
            "aggs":{
                "avg_price":{
                    "avg":{"field":"price"}
                }
            }
        }
    }
}

检索响应各字段意思

  {
    "took": 1,//这个请求耗费的时间（毫秒）
    "timed_out": false,//超时是否 一般不会超时 如果响应速度比完整性更重要可以设置超时时间  es会返回超时前收集到的数据
    "_shards": {//检索分片的信息
        "total": 5,//参与分片的数量
        "successful": 5,//有多少成功的
        "skipped": 0,
        "failed": 0//有多少失败的
    },
    "hits": {
        "total": 3,//查询总数
        "max_score": 1,//指的是所以文档中 相似度最大的值
        "hits": [
            {
                "_index": "opcmdev",//索引
                "_type": "product",//type
                "_id": "2",//id
                "_score": 1,//查询条件的相似度默认最大的排最前面  这个json没有查询条件所有都是1
                "_source": {//数据
                    "productName": "南极人（Nanjiren）床垫家纺 加厚立体床垫羽丝绒床垫床褥子学生宿舍",
                    "price": "109",
                    "remark": "不错的床垫",
                    "tags": [
                        "家具",
                        "床垫",
                        "棉花被"
                    ]
                }
            },
            {
                "_index": "opcmdev",
                "_type": "product",
                "_id": "1",
                "_score": 1,
                "_source": {
                    "productName": "法瑞思 天然椰棕床垫棕垫硬床垫定做 薄乳胶棕榈榻榻米床垫定制折叠1.2/1.5/1.8",
                    "price": "10",
                    "remark": "不错的床垫",
                    "tags": [
                        "家具",
                        "床垫",
                        "棉花被"
                    ]
                }
            },
            {
                "_index": "opcmdev",
                "_type": "product",
                "_id": "3",
                "_score": 1,
                "_source": {
                    "productName": "￥99.00志高（CHIGO）电火锅家用多功能电炒锅电煮锅不粘多用途锅电热锅 ZG-RH2",
                    "price": "99",
                    "remark": "平底锅",
                    "tags": [
                        "电器",
                        "平底锅"
                    ]
                }
            }
        ]
    }
}