IK分词器
IK ANALYZE:分析=分词的过程:字符过滤器(过滤特殊符号外加量词,的,stopword停用词)-》字符处理(词库词典)-》分词过滤(分词转换,词干转化)
1.下载
https://github.com/medcl/elasticsearch-analysis-ik/releases
注意: IK分词器插件版本必须和Elasticsearch版本一致
2、安装
进入lib目录,将下载好的文件放入
然后进入kibana检查
- 智能分词
- 最大化分词
贪心算法
#analyzer指定的是构建索引的时候分词
#search_analyzer指定的是搜索关键字时候的分词
最佳方案
索引的时候使用max_word,但是查询的时候使用smartword
点评搜索索引定义
定义字段逻辑
定义字段类型
定义字段analyzer
//定义门店索引结构
PUT /shop
{
"settings": {
"number_of_replicas": 1,
"number_of_shards": 1
},
"mappings": {
"properties": {
"id":{"type":"integer"},
"name":{"type":"text","analyzer": "ik_max_word",
"search_analyzer": "ik_smart"},
"tags":{"type":"text","analyzer": "whitespace","fielddata":true},
"location":{"type": "geo_point"},
"remark_score":{"type": "double"},
"price_per_man":{"type": "integer"},
"category_id":{"type": "integer"},
"category_name":{"type": "keyword"},
"seller_id":{"type": "integer"},
"seller_remark_score":{"type": "double"},
"seller_disabled_flag":{"type": "integer"}
}
}
}
与数据库的字段相对应
可以说是将之前的关系型数据库融入到一张表上
logstash-input-jdbc
在我们elk组件当中承担了日志收集这一功能
本质思想(管道作用)
没有追求强一致,尽量减少io操作和最大化一致
配置
在数据库中执行一下这个sql语句
和我们对应的es中建立的索引一致
进入bin目录E:\logstash-7.11.1\bin>logstash -f mysql/jdbc.conf
输入此命令启动
最后会出现这些与数据库有关的
使用get可以看到我们门店的信息已经进去了
以上是全量索引构建
再构建一个增量索引以时间轴为增量
索引
自定义一个索引
包含我们想要的打分规则
#使用function score解决排序模型
GET /shop/_search
{
"_source":"*",
"script_fields": {
"distance": {
"script": {
"source": "haversin(lat,lon,doc['location'].lat,doc['location'].lon)",
"lang": "expression",
"params": {"lat":31.37,
"lon":127.12}
}
}
},
"query": {
"function_score": {
"query": {
"bool": {
"must": [
{"match": {"name": {"query":"凯悦","boost": 0.1}}},
{"term":{"seller_disabled_flag": 0}}
]
}
},
"functions": [
{
"gauss": {
"location": {
"origin": "31.23916171,121.48789949",
"scale": "100km",
"offset": "0km",
"decay": 0.5
}
},
"weight": 9
},
{
"field_value_factor": {
"field": "remark_score"
},
"weight": 0.2
},
{
"field_value_factor": {
"field": "seller_remark_score"
},
"weight": 0.1
}
],
"score_mode": "sum",
"boost_mode": "sum"
}
},
"sort": [
{
"_score": {
"order": "desc"
}
}
]
}
java代码接入
- Node接入node client
Node接入就是将java程序看成一个node节点,所以是最重量级的一种方式 - Transport接入 transport client
transport相对来说就不是那么重量级,一般是通过各个节点的9300端口来进行信息交互,5.x之前使用比较多 - Http接入 rest client
这个方式只需要连接任意一个节点,发送http请求就可以进行交互,不需要感知网络拓扑中节点状态
推荐方法三
导入包
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-client</artifactId>
<version>7.11.1</version>
</dependency>
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>7.11.1</version>
</dependency>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>7.11.1</version>
</dependency>
在application.properties中配置
// 声明es服务地址
elasticsearch.ip=127.0.0.1:9200
然后写一个配置类接入es
package com.imooc.dianping.config;
import org.apache.http.HttpHost;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestClientBuilder;
import org.elasticsearch.client.RestHighLevelClient;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.autoconfigure.elasticsearch.rest.RestClientAutoConfiguration.RestHighLevelClientConfiguration;
import org.springframework.boot.autoconfigure.elasticsearch.rest.RestClientBuilderCustomizer;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.elasticsearch.client.RestClient;
@Configuration
public class ElasticsearchRestClient {
@Value("${elasticsearch.ip}")
String ipAddress;
@Bean(name = "highLevelClient")
public RestHighLevelClient highLevelClient(){
String [] address = ipAddress.split(":");
String ip = address[0];
int port = Integer.valueOf(address[1]);
HttpHost httpHost = new HttpHost(ip,port,"http");
return new RestHighLevelClient(RestClient.builder(new HttpHost[]{httpHost}));
}
}
然后重写shopserver中search的业务