最近做了一个爬虫项目,需要把数据存入ES中,在网上找资料的过程中挺辛苦的,大部分文章上来就是贴代码,没有讲springboot和es之间版本关系,而本身ES更新真的是快,坑是真的多(自我学习能力不强,见谅),很多方法在新版本中都被弃用,最后冷静下来,也算是终于解决了各种问题吧。
博客园...
`
@Autowired
private ElasticsearchRestTemplate elasticsearchRestTemplate;
// 高亮、分页、条件查询 从es查询
public PageInfo resultSearch(int pageNum, int pageSize, String query, String relationSiteNames) {
log.info(query);
log.info(relationSiteNames);
// (a or b) and c 条件查询
BoolQueryBuilder filterBuilder = QueryBuilders.boolQuery();
BoolQueryBuilder filterCaseBuilder = QueryBuilders.boolQuery();
filterCaseBuilder.should(QueryBuilders.matchQuery("title", query));
filterCaseBuilder.should(QueryBuilders.matchQuery("content", query));
BoolQueryBuilder filterPhoneBuilder = QueryBuilders.boolQuery();
if (StrUtil.isNotBlank(relationSiteNames)) {
// 模糊匹配
MatchPhraseQueryBuilder relationSiteNames1 = QueryBuilders.matchPhraseQuery("relationSiteNames", relationSiteNames);
filterPhoneBuilder.must(relationSiteNames1);
}
filterBuilder.must(filterCaseBuilder).must(filterPhoneBuilder);
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
boolQueryBuilder.must(filterBuilder);
// 高亮显示
HighlightBuilder.Field message = new HighlightBuilder.Field("title").preTags("<span style="color:red">").postTags("</span>");
// 分页查询
pageNum = pageNum - 1;
Pageable of = PageRequest.of(pageNum, pageSize);
// 构建查询条件
NativeSearchQuery query2 = new NativeSearchQueryBuilder()
.withQuery(boolQueryBuilder)
.withHighlightFields(message)
.withPageable(of)
.build();
// 分页查询
AggregatedPage<LawRegulationResultEs> lawRegulationResultEs = elasticsearchRestTemplate
.queryForPage(query2, LawRegulationResultEs.class, new HighlightResultMapper());
List<LawRegulationResultEs> resultEsList = lawRegulationResultEs.getContent();
resultEsList.stream().forEach(e -> e.setContent(""));
PageInfo pageHelper = new PageInfo();
pageHelper.setList(resultEsList);
pageHelper.setPages(lawRegulationResultEs.getTotalPages());
pageHelper.setTotal(lawRegulationResultEs.getTotalElements());
return pageHelper;
}
`
public class HighlightResultMapper extends DefaultResultMapper {
@Override
public <T> AggregatedPage<T> mapResults(SearchResponse response, Class<T> clazz, Pageable pageable) {
for (SearchHit hit : response.getHits()) {
Map<String, Object> sourceMap = hit.getSourceAsMap();
for (Map.Entry<String, HighlightField> entry : hit.getHighlightFields().entrySet()) {
String key = entry.getKey();
if (sourceMap.containsKey(key)) {
Text[] fragments = entry.getValue().getFragments();
sourceMap.put(key, transTextArrayToString(fragments));
}
}
hit.sourceRef(new ByteBufferReference(ByteBuffer.wrap(JSONObject.toJSONString(sourceMap).getBytes())));
}
return super.mapResults(response, clazz, pageable);
}
private String transTextArrayToString(Text[] fragments) {
if (null == fragments) {
return "";
}
StringBuffer buffer = new StringBuffer();
for (Text fragment : fragments) {
buffer.append(fragment.string());
}
return buffer.toString();
}