ElasticSearch笔记

EelasticSearch安装

Docker安装

docker run -d --name es7  --restart=always -e ES_JAVA_POTS="-Xms256m -Xmx256m" -e "discovery.type=single-node" -v /opt/es7/data/:/usr/share/elasticsearch/data  -p 9200:9200 -p 9300:9300 elasticsearch:7.14.0

客户端UI工具,Edge浏览器扩展

在这里插入图片描述

分词器安装

  • 在 Elasticsearch 的 IK Analyzer 中,ik_smart和 ik_max_word 是IK 分词器针对中文分词提供的两种策略
  • ik_max_word 更倾向于全面细致的分词,而 ik_smart 则偏向于精简和精准的分词。在实际应用场景中,选择哪种模式取决于项目的具体需求,如是否需要扩大搜索覆盖面还是提高搜索准确性。

下载:elasticsearch-analysis-ik

#第一步 copy 插件到容器
docker cp /opt/es7/elasticsearch-analysis-ik-7.14.0.zip 容器id:/usr/share/elasticsearch

#第二步进入你的容器
docker exec -it 容器id /bin/bash

#第三步执行如下命令,安装插件,中间会提示 Y or N,直接写 Y ,回车即可
elasticsearch-plugin install file:\/usr/share/elasticsearch/elasticsearch-analysis-ik-7.14.0.zip

#第四步退出容器
exit

#第五步重启容器
docker restart 容器ID

使用客户端查看

在这里插入图片描述
在这里插入图片描述

原理

在这里插入图片描述

EelasticSearch使用

Easy-ES介绍

easy-es官网

导入依赖包

 <dependency>
     <groupId>org.springframework.boot</groupId>
     <artifactId>spring-boot-starter-web</artifactId>
     <exclusions>
         <exclusion>
             <groupId>org.elasticsearch.client</groupId>
             <artifactId>elasticsearch-rest-high-level-client</artifactId>
         </exclusion>
         <exclusion>
             <groupId>org.elasticsearch</groupId>
             <artifactId>elasticsearch</artifactId>
         </exclusion>
     </exclusions>
</dependency>
<dependency>
     <groupId>cn.easy-es</groupId>
     <artifactId>easy-es-boot-starter</artifactId>
     <version>2.0.0-beta1</version>
</dependency>
<dependency>
     <groupId>org.elasticsearch.client</groupId>
     <artifactId>elasticsearch-rest-high-level-client</artifactId>
     <version>7.14.0</version>
</dependency>
<dependency>
     <groupId>org.elasticsearch</groupId>
     <artifactId>elasticsearch</artifactId>
     <version>7.14.0</version>
</dependency>

在对像属性上添加注解

开启mapper扫描

EsMapperScan("cn.easy-es-mapper")
public class App{
    // 省略其它...
}

注解 | esay-es

@IndexName( value = "product",aliasName = "es_product")
public class EsProduct {

    @IndexId(type = IdType.CUSTOMIZE)
    private Integer id;
    @IndexField( fieldType= FieldType.TEXT, analyzer = Analyzer.IK_MAX_WORD,searchAnalyzer = Analyzer.IK_MAX_WORD)
    private String name;
    @IndexField( fieldType= FieldType.INTEGER)
    private Integer categoryId;
    @IndexField( fieldType= FieldType.DOUBLE) // 12.56
    private BigDecimal price;
    @IndexField( fieldType= FieldType.TEXT, analyzer = Analyzer.IK_MAX_WORD)
    private String brief;
    @IndexField( fieldType= FieldType.KEYWORD)
    private String img;
    @IndexField( fieldType= FieldType.TEXT, analyzer = Analyzer.IK_MAX_WORD)
    private List<String> tags;
    @IndexField( fieldType= FieldType.INTEGER) //198
    private Integer highOpinion;
    @IndexField( fieldType= FieldType.INTEGER)
    private Integer salesVolume;
    @IndexField( fieldType= FieldType.DATE)
    private LocalDateTime productionDate;
}

新建Mapper类,类似Mybatis的dao

import cn.easyes.core.core.BaseEsMapper;
public interface ProductMapper  extends BaseEsMapper<Product> {

}

配置ES

# Easy-Es配置部分
easy-es:
  # 启用Easy-Es功能
  enable: true
  # 设置Elasticsearch服务器地址和端口
  address: 192.168.23.27:9200
  # 全局配置项,设置是否打印执行的DSL语句(便于调试)
  global-config:
    print-dsl: true

创建、删除、查询索引

分词&模糊匹配|Esay-Es

新增

 	@Test
    void insert() {
        EsProduct esProduct = EsProduct.builder()
                .id(1)
                .name("小米")
                .img("图片地址")
                .brief("小米(MI)Redmi Note12 5G 120Hz OLED屏幕 骁龙4移动平台 5000mAh长续航 8GB+128GB子夜黑 小米红米")
                .price(new BigDecimal(18))
                .categoryId(1)
                .highOpinion(20)
                .productionDate(LocalDateTime.now())
                .salesVolume(99)
                .tags(CollUtil.newArrayList("120高刷","舒适护眼"))
                .build();
        esMapper.insert(esProduct);
    }
	@Test
    void insertBatch() {
        delete();
        List<EsProduct> esProducts = CollUtil.newArrayList();
        for (int i = 1; i <= 10; i++) {
            EsProduct esProduct = EsProduct.builder()
                    .id(i)
                    .name("小米"+i)
                    .img("图片地址"+i)
                    .brief("小米(MI)Redmi Note"+i+" 5G 120Hz OLED屏幕 骁龙4移动平台 5000mAh长续航 8GB+128GB子夜黑 小米红米")
                    .price(new BigDecimal(500.36+i))
                    .categoryId(1)
                    .highOpinion(100+i)
                    .productionDate(LocalDateTime.now())
                    .salesVolume(200+i)
                    .tags(CollUtil.newArrayList("12"+i+"高刷","舒适护眼"))
                    .build();
            esProducts.add(esProduct);
                }

        esMapper.insertBatch(esProducts);
    }

修改

	@Test
    void update() {
        EsProduct esProduct = EsProduct.builder()
                .id(1)
                .name("su7")
                .img("图片地址")
                .brief("小米汽车")
                .price(new BigDecimal(18))
                .categoryId(9)
                .highOpinion(20)
                .productionDate(LocalDateTime.now())
                .salesVolume(99)
                .tags(CollUtil.newArrayList("120高刷","舒适护眼"))
                .build();
        Integer integer = esMapper.updateById(esProduct);
    }

删除

	@Test
    void delete() {
    	// 批量删除
        Integer batchIds = esMapper.deleteBatchIds(CollUtil.newArrayList(1, 2, 3, 4, 5, 6, 7, 8, 9, 10));
		// 根据id删除
        Integer integer = esMapper.deleteById(1);
    }

查询

	@Test
    void eqSelect() {
        LambdaEsQueryWrapper<EsProduct> queryWrapper = new LambdaEsQueryWrapper<EsProduct>();
        queryWrapper.eq("id",1);
        List<EsProduct> list = esMapper.selectList(queryWrapper);
    }

	@Test
    void selectMatchAll() {
        LambdaEsQueryWrapper<EsProduct>  queryWrapper = new LambdaEsQueryWrapper<EsProduct>();
        queryWrapper.queryStringQuery("汽车之家"); //所有字段都去匹配
        List<EsProduct> list = esMapper.selectList(queryWrapper);
    }

	@Test
    void inSelect() {
        LambdaEsQueryWrapper<EsProduct>  queryWrapper = new LambdaEsQueryWrapper<>();
        queryWrapper.in(EsProduct::getCategoryId,1,9);
        List<EsProduct> list = esMapper.selectList(queryWrapper);
    }

	@Test
    void andSelect() {
        LambdaEsQueryWrapper<EsProduct>  queryWrapper = new LambdaEsQueryWrapper<>();
        queryWrapper.in("categoryId",1,9); //where categroyId in (1,9) and ( name like '%汽车%' or brief like '%汽车%')
        queryWrapper.and(
                w->w.match(EsProduct::getBrief,"汽车",1.0F)
                        .or().match(EsProduct::getName,"汽车",2.0F));
        List<EsProduct> list = esMapper.selectList(queryWrapper);
    }

	@Test
    void orderBySelect() {
        LambdaEsQueryWrapper<EsProduct>  queryWrapper = new LambdaEsQueryWrapper<>();
        queryWrapper.in("categoryId",1,9); //where categroyId in (1,9) and ( name like '%汽车%' or brief like '%汽车%') order by salesVolume desc
        queryWrapper.and(
                w->w.match(EsProduct::getBrief,"高刷",1.0F)
                        .or().match(EsProduct::getName,"高刷",2.0F)
                        .or().match(EsProduct::getTags,"高刷",1.0F));
        queryWrapper.orderByDesc(EsProduct::getSalesVolume);
        EsPageInfo<EsProduct> esProductEsPageInfo = esMapper.pageQuery(queryWrapper, 1, 2);
    }

综合使用(商品查询)

  • es商品对象实体类
@IndexName(value = "product", aliasName = "es_product")
@Data
@Builder
@AllArgsConstructor
@NoArgsConstructor
public class EsProduct {
     @IndexId(type = IdType.CUSTOMIZE)
     private Integer id;
     @IndexField(fieldType = FieldType.TEXT, analyzer = Analyzer.IK_SMART, searchAnalyzer = Analyzer.IK_MAX_WORD)
     private String name;
     //FieldType.KEYWORD代表本身就是一个关键字(不可再分的关键字) 不需要再分了
     @IndexField(fieldType = FieldType.KEYWORD)
     private String brand;   //格力 美的 美特斯邦威
     @IndexField(fieldType = FieldType.TEXT, analyzer = Analyzer.IK_SMART, searchAnalyzer = Analyzer.IK_MAX_WORD)
     private String subName;
     @IndexField(fieldType = FieldType.INTEGER)
     private Integer categoryId;
     @IndexField(fieldType = FieldType.DOUBLE)
     private BigDecimal price;
     @IndexField(fieldType = FieldType.TEXT, analyzer = Analyzer.IK_SMART, searchAnalyzer = Analyzer.IK_MAX_WORD)
     private String brief;
     //    @IndexField(exist=false)
     @IndexField(fieldType = FieldType.NONE)
     private String img;
     @IndexField(fieldType = FieldType.TEXT, analyzer = Analyzer.IK_SMART, searchAnalyzer = Analyzer.IK_MAX_WORD)
     private List<String> tags;
     @IndexField(fieldType = FieldType.LONG)
     private Integer highOpinion;    //好评
     @IndexField(fieldType = FieldType.LONG)
     private Integer salesVolume;    //销量
     //排序
     @IndexField(fieldType = FieldType.DATE)
     private LocalDateTime productionDate;
 }


  @Data
  public class EsProductQuery {
 
      private Integer categoryId;
      private String keyWord;
      private String tags;
      private Integer highOpinion;
      private Integer salesVolume;
      private LocalDateTime productionDate;
      private Integer pageSize;
      private Integer pageNum;
 
  }
  • es查询商品列表接口
public interface EsProductMapper extends BaseEsMapper<EsProduct> {
}

public Map<String, Object> ESSelect(EsProductQuery query) {

         LambdaEsQueryWrapper<EsProduct> wrapper = new LambdaEsQueryWrapper<>();

         if (query.getCategoryId() != null) {
             wrapper.eq(EsProduct::getCategoryId, query.getCategoryId());
         }
         if (query.getTags() != null) {
             String tags = query.getTags();
             wrapper.match(EsProduct::getTags, tags);
         }

         if (query.getKeyWord() != null) {
             String keyWord = query.getKeyWord();
             wrapper.and(w->w.match(EsProduct::getName,keyWord,5f).or().match(EsProduct::getSubName,keyWord,3f).or().match(EsProduct::getBrief, keyWord,1f));
         }


         if (query.getSalesVolume() != null) {
             wrapper.orderByDesc(EsProduct::getSalesVolume);
         }

         if (query.getHighOpinion() != null) {
             wrapper.orderByDesc(EsProduct::getHighOpinion);
         }

         EsPageInfo<EsProduct> esProductEsPageInfo = esProductMapper.pageQuery(wrapper, query.getPageNum(), query.getPageSize());
         long total = esProductEsPageInfo.getTotal();
         List<EsProduct> list = esProductEsPageInfo.getList();

         Map<String, Object> map = new HashMap<>();
         map.put("items", list);
         map.put("total", total);
         String source = esProductMapper.getSource(wrapper);
         System.out.println(source);
         return map;
     }

ELK(EFK)日志采集系统

在这里插入图片描述

MDC

使用

在这里插入图片描述

配置TraceId 过滤器

@Order(1)

@WebFilter(urlPatterns = "/*",filterName = "traceIdFilter")
public class TraceIdFilter implements Filter {

    public final static String MDC_TRACE_ID = "traceId";

    @Override
    public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException {
        HttpServletRequest httpRequest = (HttpServletRequest) request;
        String traceId = httpRequest.getHeader(MDC_TRACE_ID);
        if (StringUtils.isBlank(traceId)) {
            traceId = IdUtil.fastSimpleUUID();;
        }
        MDC.put(MDC_TRACE_ID, traceId);

        chain.doFilter(request, response);
    }
}

启动类开启

@SpringBootApplication

@ServletComponentScan
public class OpenApp {
    public static void main(String[] args) {
        SpringApplication.run(OpenApp.class, args);
    }

}

配置文件配置日志输出格式

# Spring Boot 日志配置
logging:
  # 控制台日志输出格式
  pattern:
    console: "%d{yyyy-MM-dd HH:mm:ss.SSS} %clr(%-5level) %clr([%X{traceId}]) %clr(${PID:-}) --- %clr(%logger{50}) - %m%n"

# 注释说明:
# %d{yyyy-MM-dd HH:mm:ss.SSS}: 时间戳,格式为年-月-日 时:分:秒:毫秒
# %-5level: 日志级别,左对齐且至少占用5个字符宽度
# %clr(...): 使用颜色编码(如果支持)
# [%X{traceId}]: Mapped Diagnostic Context中的traceId,用于记录跟踪请求的唯一标识
# ${PID:-}: 当前进程ID,如果没有则显示空字符串
# ---: 分隔符
# %logger{50}: 日志器名称,最多显示50个字符
# %m: 日志消息内容
# %n: 换行符

在这里插入图片描述
在这里插入图片描述

EFK安装与使用

docker-compose.yml

version: '3'
services:
  elasticsearch:
    image: elasticsearch:7.14.0
    ports:
      - "9200:9200"
    environment:
      - discovery.type=single-node
      - ES_JAVA_OPTS=-Xmx256m -Xms256m

  kibana:
    image: kibana:7.14.0
    ports:
      - "5601:5601"
    depends_on:
      - elasticsearch
    environment:
      - ELASTICSEARCH_HOSTS=http://elasticsearch:9200
      - I18N_LOCALE="zh-CN"

  filebeat:
    image:  elastic/filebeat:7.14.0
    volumes:
      - ./filebeat.yml:/usr/share/filebeat/filebeat.yml
      - /logs:/usr/share/filebeat/logs
    depends_on:
      - elasticsearch
    command: ["-e"]

filebeat.yml

filebeat.inputs:
- type: log
  paths:
    - /usr/share/filebeat/logs/*.log
  processors:
    - dissect:
        tokenizer: "%{appName} [%{logTime}] %{logLevel} [%{thread}] [%{traceId}] %{logger} [%{line}] %{message}"
        field: "message"
        target_prefix: "logs"
output.elasticsearch:
  hosts: ["elasticsearch:9200"]
  indices:
    - index: "app-logs-%{+YYYY.MM.dd}"

日志配置

 #日志配置
# Console Appender
logging.pattern.console=${spring.application.name} %d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] [%X{traceId}] %-5level %logger{36} --- %msg%n

# File Appender
logging.file.name=/logs/${spring.application.name}.log
logging.pattern.file=${spring.application.name} [%date] %level [%thread] [%X{traceId}] %logger{36} [%file:%line] %msg%n
logging.file.max-size = 100MB
logging.file.max-history = 3
logging.level.root=error
logging.level.com.beiyou=debug

如果容器中的时区不正确,请加入如下语句调整时区

#为防止docker容器中的时区错乱,这里设置容器内部时区
ENV TZ=Asia/Shanghai
RUN ln -sf /usr/share/zoneinfo/$TZ /etc/localtime    && echo $TZ > /etc/timezone
  • 2
    点赞
  • 3
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值