ES Document API之多文档API

多文档API

• 多获取API Get API

#获取一个类型的多个文档，有多种API写法，如下：
#1
curl -XGET 'localhost:9200/_mget?pretty' -H 'Content-Type: application/json' -d'
{
    "docs" : [
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "1"
        },
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "2"
        }
    ]
}
'

#2
curl -XGET 'localhost:9200/test/_mget?pretty' -H 'Content-Type: application/json' -d'
{
    "docs" : [
        {
            "_type" : "type",
            "_id" : "1"
        },
        {
            "_type" : "type",
            "_id" : "2"
        }
    ]
}
'
#3
curl -XGET 'localhost:9200/test/type/_mget?pretty' -H 'Content-Type: application/json' -d'
{
    "docs" : [
        {
            "_id" : "1"
        },
        {
            "_id" : "2"
        }
    ]
}
'
#4
curl -XGET 'localhost:9200/test/type/_mget?pretty' -H 'Content-Type: application/json' -d'
{
    "ids" : ["1", "2"]
}
'

#_type字段可选，如果没有指定type，则默认第一个类型中满足条件的id文档作为返回结果，下面的例子中，返回两个相同的结果
curl -XGET 'localhost:9200/test/_mget?pretty' -H 'Content-Type: application/json' -d'
{
    "ids" : ["1", "1"]
}
'

#可以明确指定不同的类型
curl -XGET 'localhost:9200/test/_mget/?pretty' -H 'Content-Type: application/json' -d'
{
  "docs" : [
        {
            "_type":"typeA",
            "_id" : "1"
        },
        {
            "_type":"typeB",
            "_id" : "1"
        }
    ]
}
'

#过滤_source
curl -XGET 'localhost:9200/_mget?pretty' -H 'Content-Type: application/json' -d'
{
    "docs" : [
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "1",
            "_source" : false
        },
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "2",
            "_source" : ["field3", "field4"]
        },
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "3",
            "_source" : {
                "include": ["user"],
                "exclude": ["user.location"]
            }
        }
    ]
}
'

#指定stored_field字段返回 1 返回 field1，field2,2返回field3，field4
curl -XGET 'localhost:9200/test/type/_mget?stored_fields=field1,field2&pretty' -H 'Content-Type: application/json' -d'
{
    "docs" : [
        {
            "_id" : "1" 
        },
        {
            "_id" : "2",
            "stored_fields" : ["field3", "field4"] 
        }
    ]
}
'

#指定routing
curl -XGET 'localhost:9200/_mget?routing=key1&pretty' -H 'Content-Type: application/json' -d'
{
    "docs" : [
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "1",
            "routing" : "key2"
        },
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "2"
        }
    ]
}
'

• 批量API Bulk API 提高索引效率

批量操作在写法上需要注意的事项：换行 结尾； 回车 ；开始 Content-Type头部应该被设置为application/x-ndjson；

支持的参数：version_type;routing;wait_for_active_shards;refresh;

update操作支持的动作参数：retry_on_conflict;doc;doc_as_upsert;script;lang;source.

#批量操作数据结构

action_and_meta_data  n
optional_source  n
action_and_meta_data  n
optional_source  n
....
action_and_meta_data  n
optional_source  n

curl -XPOST 'localhost:9200/_bulk?pretty' -H 'Content-Type: application/json' -d'
{ "index" : { "_index" : "test", "_type" : "type1", "_id" : "1" } }
{ "field1" : "value1" }
{ "delete" : { "_index" : "test", "_type" : "type1", "_id" : "2" } }
{ "create" : { "_index" : "test", "_type" : "type1", "_id" : "3" } }
{ "field1" : "value3" }
{ "update" : {"_id" : "1", "_type" : "type1", "_index" : "test"} }
{ "doc" : {"field2" : "value2"} }
'

curl -XPOST 'localhost:9200/_bulk?pretty' -H 'Content-Type: application/json' -d'
{ "update" : {"_id" : "1", "_type" : "type1", "_index" : "index1", "retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"} }
{ "update" : { "_id" : "0", "_type" : "type1", "_index" : "index1", "retry_on_conflict" : 3} }
{ "script" : { "source": "ctx._source.counter += params.param1", "lang" : "painless", "params" : {"param1" : 1}}, "upsert" : {"counter" : 1}}
{ "update" : {"_id" : "2", "_type" : "type1", "_index" : "index1", "retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"}, "doc_as_upsert" : true }
{ "update" : {"_id" : "3", "_type" : "type1", "_index" : "index1", "_source" : true} }
{ "doc" : {"field" : "value"} }
{ "update" : {"_id" : "4", "_type" : "type1", "_index" : "index1"} }
{ "doc" : {"field" : "value"}, "_source": true}
'

• Term vectors 返回特定文档信息中的术语信息和统计信息

#示例
curl -XGET 'localhost:9200/twitter/tweet/1/_termvectors?pretty'

  #可以指定要检索的字段

  curl -XGET 'localhost:9200/twitter/tweet/1/_termvectors?fields=message&pretty'

返回字段包含：term infomation,term statistics,field statistics。默认返回：term infomation和field statistics。

term infomation:1.字段的term频率；2.term 位置；3.开始和结束偏移量；4.term 有效负载      即使没保存也可以计算。

term statistics :1.术语在总文档中出现的频次 2.包含当前术语的总文档数

field statistics ： 1.包含该字段的文档数doc_count  2.该字段中所有term的文档总频次 sum_doc_freq 3.该term中每个term的总频次 sum_ttf

• multi term vectors 一次性取回多个termvectors

curl -XPOST 'localhost:9200/_mtermvectors?pretty' -H 'Content-Type: application/json' -d'
{
   "docs": [
      {
         "_index": "twitter",
         "_type": "tweet",
         "_id": "2",
         "term_statistics": true
      },
      {
         "_index": "twitter",
         "_type": "tweet",
         "_id": "1",
         "fields": [
            "message"
         ]
      }
   ]
}
'

• Search API

查询API通常是多文档 多类型的，除explain api

查询API可指定多个routing，用“，”隔开

自适应副本选择：代替循环查询副本的方式，将请求发送到被认为是最佳的分片上：

• Response time of past requests between the coordinating node and the node containing the copy of the data
• Time past search requests took to execute on the node containing the data
• The queue size of the search threadpool on the node containing the data

统计信息组：search 与信息统计组关联，每个统计组包含其统计聚合

全局搜索超时：search.default_search_timeout  在集群更新中设置，设置为-1时，即不超时

搜索任务取消：task cancell中取消搜索任务

#指定routing
curl -XPOST 'localhost:9200/twitter/tweet/_search?routing=kimchy&pretty' -H 'Content-Type: application/json' -d'
{
    "query": {
        "bool" : {
            "must" : {
                "query_string" : {
                    "query" : "some query string here"
                }
            },
            "filter" : {
                "term" : { "user" : "kimchy" }
            }
        }
    }
}
'

#动态集群设置，自适应分片选择参数为true，默认为false
curl -XPUT 'localhost:9200/_cluster/settings?pretty' -H 'Content-Type: application/json' -d'
{
    "transient": {
        "cluster.routing.use_adaptive_replica_selection": true
    }
}
'

#与统计信息组做关联
curl -XPOST 'localhost:9200/_search?pretty' -H 'Content-Type: application/json' -d'
{
    "query" : {
        "match_all" : {}
    },
    "stats" : ["group1", "group2"]
}
'

• 通过查询API更新
• Reindex API