Script
-- 使用painless脚本
# 数量减一, 安全的
POST produce/_update/1 { "script": { "source": "ctx._source.price-=1" } } # 数量减一, 安全的<简写> POST produce/_update/1 {"script": "ctx._source.price-=1"}
查询所有的价格
GET produce/_search { "script_fields": { "test_field": { "script": { "lang": "painless", "source": "doc['price'].value" } } } }
给数组中添加元素
POST product/_update/2 { "script": { "lang": "painless", "source": "ctx._source.tags.add('无线充电')" } }
传参的方式，给数组中添加元素
POST product/_update/2 { "script": { "lang": "painless", "source": "ctx._source.tags.add(params.tag_name)", "params": { "tag_name": "新增" } } }
删除数据
POST product/_update/3 { "script": { "lang": "painless", "source": "ctx.op='delete'" } }
如果数据存在,执行部分update操作,如果数据不存在,那么执行create操作
POST produce/_update/2 { "script": { "source": "ctx._source.price += params.paraml", "lang": "painless", "params": { "paraml": 100 } }, "upsert": { "name": "xiaoke", "price": 2000 } }
_bulk更新
POST _bulk { "update" : { "_id" : "5", "_index" : "produce", "retry_on_conflict" : 3} } { "script" : { "source": "ctx._source.price += params.param1", "lang" : "painless", "params" : {"param1" : 100}}, "upsert" : {"price" : 1999}}
打8折价格
GET produce/_search { "script_fields": { "discount_price": { "script": { "lang": "painless", "source": "doc['price'].value * params.discount", "params": { "discount" : 0.8 } } } } } GET produce/_search { "script_fields": { "discount_price": { "script": { "lang": "painless", "source": "doc.price.value * params.discount", "params": { "discount": 0.8 } } } } }
原始价格 和 多个打折价格
GET produce/_search { "script_fields": { "init_price": { "script": { "lang": "painless", "source": "doc['price'].value" } }, "discount_price":{ "script":{ "lang": "painless", "source": "[doc['price'].value * params.discount_8,doc['price'].value * params.discount_7,doc['price'].value * params.discount_6,doc['price'].value * params.discount_5]", "params": { "discount_8": 0.8, "discount_7": 0.7, "discount_6": 0.6, "discount_5": 0.5 } } } } }
缓存方式查询
# /_scripts/{id} 类似存储过程 计算折扣 作用域为整个集群 # 默认缓存大小是100MB 没有过期时间 可以手工设置过期时间script.cache.expire 通过script.cache.max_size设置缓存大小 脚本最大64MB 创建 POST _scripts/calculate-discount { "script": { "lang": "painless", "source": "doc['price'].value * params.discount" } } 查看 GET _scripts/calculate-discount 使用 GET produce/_search { "script_fields": { "discount_price": { "script": { "id":"calculate-discount", "params": { "discount": 0.8 } } } } } 删除 DELETE _scripts/calculate-discount
Date
PUT /one/_doc/2 { "name": "xiaoke", "createtime": "2021-09-06" } # year、month、dayOfMonth、dayOfWeek、dayOfYear、hour、minute、second、nano GET one/_search { "script_fields": { "test_year": { "script": { "source": "doc.createtime.value.year" } } } }
复杂操作，多个column更新
POST produce/_update/1 { "script": { "lang": "painless", "source": """ ctx._source.name += params.name; ctx._source.price -= 1 """, "params": { "name": "无线充电", "price": "1" } } }
正则匹配, 开启： script.painless.regex.enabled: true
POST produce/_update/1 { "script": { "lang": "painless", "source": """ if (ctx._source.name =~ /[sS]*phone[sS]*/) { ctx._source.name += "***|"; } else { ctx.op = "noop"; } """ } } POST one/_update/1 { "script": { "lang": "painless", "source": """ if (ctx._source.createtime ==~ /[0-9]{4}-[0-9]{2}-[0-9]{2}/) { ctx._source.name += "|***"; } else { ctx.op = "noop"; } """ } }
统计所有小于1000商品tag的 数量 不考虑去重
GET /produce/_search { "query": { "bool": { "filter": [ { "range": { "price": { "lt": 1000 } } } ] } }, "aggs": { "tag_agg_group": { "sum": { "script": { "lang": "painless", "source": """ int total = 0; for (int i = 0; i < doc['tags'].length; i++) { total++ } return total; """ } } } }, "size": 0 }
练习题
测试数据： PUT /test_indexc/_bulk?refresh {"index":{"_id":1}} {"ajbh": "12345","ajmc": "立案案件","lasj": "2020/05/21 13:25:23","jsbax_sjjh2_xz_ryjbxx_cleaning": [{"XM": "张三","NL": "30","SF": "男"},{"XM": "李四","NL": "31","SF": "男"},{"XM": "王五","NL": "30","SF": "女"},{"XM": "赵六","NL": 23,"SF": "男"}]} {"index":{"_id":2}} {"ajbh": "563245","ajmc": "结案案件","lasj": "2020/05/21 13:25:23","jsbax_sjjh2_xz_ryjbxx_cleaning": [{"XM": "张三2","NL": "30","SF": "男"},{"XM": "李四2","NL": "31","SF": "男"},{"XM": "王五2","NL": "30","SF": "女"},{"XM": "赵六2","NL": 23,"SF": "女"}]} {"index":{"_id":3}} {"ajbh": "12345","ajmc": "立案案件","lasj": "2020/05/21 13:25:23","jsbax_sjjh2_xz_ryjbxx_cleaning": [{"XM": "张三3","NL": "30","SF": "男"},{"XM": "李四3","NL": "31","SF": "男"},{"XM": "王五3","NL": "30","SF": "女"},{"XM": "赵六3","NL": 23,"SF": "男"}]} 求： 所有案件中男性的数量 GET /test_index1/_search { "aggs": { "sum_person": { "sum": { "script": { "lang": "painless", "source": """ int total = 0; for (int i = 0; i < params['_source']['jsbax_sjjh2_xz_ryjbxx_cleaning'].length; i++) { if (params['_source']['jsbax_sjjh2_xz_ryjbxx_cleaning'][i]['SF'] == '男') { total += 1; } } return total; """ } } } }, "size": 0 }
ES写入原理
| 写入步骤 | ||
|---|---|---|
| 1.client请求 | ||
| 2. a.内存Buffer b.持久化到translog文件 | 1.只是负责写入，很快会写入index segment file中 2.通过refresh写入到segment中 refresh | |
| 3.index segment | 1.用来增删改查的缓存 flush、a.刷到os cache b.触发commit | |
| 4.OS cache | fsync | |
| 5.OS disk |
refresh
1.将内存Buffer中导数据刷新到segment中 2.自动刷新 PUT /index {"settings":{"refresh_interval": "10s"}} 3.手动刷新 POST /index/refresh
流程
refresh行为会立即把缓存中的文档写入segment中，但是此时新创建的segment是写在文件系统的缓存中的，如果出现断电等异常，name这部分数据就会丢失。所以es会定期执行flush操作，将缓存中segment全部写入磁盘并确保写入成功，同时创建一个commit point，整个过程就是一个完整的commit过程。
flush
1.手动执行 POST /index/_flush 2.自动执行 触发flush:当文件大小达到阈值或者一定时间(默认30分钟) a.将数据刷到os cache b.同步创建commit c.fsync d.清空translog e.如果d之前出现了关机，内存数据丢失，会从translog文件加载 segment删除步骤 1.选择相似segment合并 2.flush操作 3.创建新的commit piont标记新的segment,删除旧的标记 4.将新的segment搜索状态打开 5.删除旧的segment文件