第一章es安装和简单配置

1、官网下载es安装包

 地址：https://www.elastic.co/downloads/elasticsearch

解压目录结构如下

2、进入目录执行elasticsearch-7.10.2 执行命令 bin/elasticsearch

[2021-03-09T11:01:33,864][INFO ][o.e.n.Node               ] [localhost] version[7.10.2], pid[12496], build[default/tar/747e1cc71def077253878a59143c1f785afa92b9/2021-01-13T00:42:12.435326Z], OS[Mac OS X/10.15.2/x86_64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_201/25.201-b09]
[2021-03-09T11:01:33,873][INFO ][o.e.n.Node               ] [localhost] JVM home [/Library/Java/JavaVirtualMachines/jdk1.8.0_201.jdk/Contents/Home/jre], using bundled JDK [false]
[2021-03-09T11:01:33,874][INFO ][o.e.n.Node               ] [localhost] JVM arguments [-Xshare:auto, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,JRE, -Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.io.tmpdir=/var/folders/fr/b7sn9sy57hj566s46tc_45w80000gn/T/elasticsearch-8746849917296871633, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -XX:MaxDirectMemorySize=536870912, -Des.path.home=/Users/wangyuanyuan/tool/elasticsearch-7.10.2, -Des.path.conf=/Users/wangyuanyuan/tool/elasticsearch-7.10.2/config, -Des.distribution.flavor=default, -Des.distribution.type=tar, -Des.bundled_jdk=true]
[2021-03-09T11:01:36,856][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [aggs-matrix-stats]
[2021-03-09T11:01:36,856][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [analysis-common]
[2021-03-09T11:01:36,857][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [constant-keyword]
[2021-03-09T11:01:36,857][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [flattened]
[2021-03-09T11:01:36,857][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [frozen-indices]
[2021-03-09T11:01:36,857][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [ingest-common]
[2021-03-09T11:01:36,857][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [ingest-geoip]
[2021-03-09T11:01:36,858][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [ingest-user-agent]
[2021-03-09T11:01:36,858][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [kibana]
[2021-03-09T11:01:36,858][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [lang-expression]
[2021-03-09T11:01:36,858][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [lang-mustache]
[2021-03-09T11:01:36,858][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [lang-painless]
[2021-03-09T11:01:36,859][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [mapper-extras]
[2021-03-09T11:01:36,859][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [mapper-version]
[2021-03-09T11:01:36,859][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [parent-join]
[2021-03-09T11:01:36,859][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [percolator]
[2021-03-09T11:01:36,860][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [rank-eval]
[2021-03-09T11:01:36,860][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [reindex]
[2021-03-09T11:01:36,860][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [repositories-metering-api]
[2021-03-09T11:01:36,860][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [repository-url]
[2021-03-09T11:01:36,860][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [search-business-rules]
[2021-03-09T11:01:36,861][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [searchable-snapshots]
[2021-03-09T11:01:36,861][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [spatial]
[2021-03-09T11:01:36,861][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [transform]
[2021-03-09T11:01:36,861][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [transport-netty4]
[2021-03-09T11:01:36,861][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [unsigned-long]
[2021-03-09T11:01:36,861][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [vectors]
[2021-03-09T11:01:36,862][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [wildcard]
[2021-03-09T11:01:36,862][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [x-pack-analytics]
[2021-03-09T11:01:36,862][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [x-pack-async]
[2021-03-09T11:01:36,862][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [x-pack-async-search]
[2021-03-09T11:01:36,862][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [x-pack-autoscaling]
[2021-03-09T11:01:36,862][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [x-pack-ccr]
[2021-03-09T11:01:36,863][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [x-pack-core]
[2021-03-09T11:01:36,863][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [x-pack-data-streams]
[2021-03-09T11:01:36,863][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [x-pack-deprecation]
[2021-03-09T11:01:36,863][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [x-pack-enrich]
[2021-03-09T11:01:36,863][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [x-pack-eql]
[2021-03-09T11:01:36,863][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [x-pack-graph]
[2021-03-09T11:01:36,864][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [x-pack-identity-provider]
[2021-03-09T11:01:36,864][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [x-pack-ilm]
[2021-03-09T11:01:36,864][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [x-pack-logstash]
[2021-03-09T11:01:36,864][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [x-pack-ml]
[2021-03-09T11:01:36,864][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [x-pack-monitoring]
[2021-03-09T11:01:36,864][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [x-pack-ql]
[2021-03-09T11:01:36,864][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [x-pack-rollup]
[2021-03-09T11:01:36,865][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [x-pack-security]
[2021-03-09T11:01:36,865][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [x-pack-sql]
[2021-03-09T11:01:36,865][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [x-pack-stack]
[2021-03-09T11:01:36,865][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [x-pack-voting-only-node]
[2021-03-09T11:01:36,865][INFO ][o.e.p.PluginsService     ] [localhost] loaded module [x-pack-watcher]
[2021-03-09T11:01:36,866][INFO ][o.e.p.PluginsService     ] [localhost] no plugins loaded
[2021-03-09T11:01:36,916][INFO ][o.e.e.NodeEnvironment    ] [localhost] using [1] data paths, mounts [[/ (/dev/disk1s5)]], net usable_space [101.9gb], net total_space [233.4gb], types [apfs]
[2021-03-09T11:01:36,916][INFO ][o.e.e.NodeEnvironment    ] [localhost] heap size [989.8mb], compressed ordinary object pointers [true]
[2021-03-09T11:01:37,040][INFO ][o.e.n.Node               ] [localhost] node name [localhost], node ID [O29ASbxJQYqCAqMmqXxg3w], cluster name [elasticsearch], roles [transform, master, remote_cluster_client, data, ml, data_content, data_hot, data_warm, data_cold, ingest]
[2021-03-09T11:01:40,465][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [localhost] [controller/12529] [Main.cc@114] controller (64 bit): Version 7.10.2 (Build 40a3af639d4698) Copyright (c) 2021 Elasticsearch BV
[2021-03-09T11:01:41,024][INFO ][o.e.x.s.a.s.FileRolesStore] [localhost] parsed [0] roles from file [/Users/wangyuanyuan/tool/elasticsearch-7.10.2/config/roles.yml]
[2021-03-09T11:01:42,491][INFO ][o.e.t.NettyAllocator     ] [localhost] creating NettyAllocator with the following configs: [name=unpooled, suggested_max_allocation_size=1mb, factors={es.unsafe.use_unpooled_allocator=null, g1gc_enabled=false, g1gc_region_size=0b, heap_size=989.8mb}]
[2021-03-09T11:01:42,573][INFO ][o.e.d.DiscoveryModule    ] [localhost] using discovery type [zen] and seed hosts providers [settings]
[2021-03-09T11:01:43,150][WARN ][o.e.g.DanglingIndicesState] [localhost] gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually
[2021-03-09T11:01:43,595][INFO ][o.e.n.Node               ] [localhost] initialized
[2021-03-09T11:01:43,595][INFO ][o.e.n.Node               ] [localhost] starting ...
[2021-03-09T11:01:43,787][INFO ][o.e.t.TransportService   ] [localhost] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
[2021-03-09T11:01:44,101][WARN ][o.e.b.BootstrapChecks    ] [localhost] the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured
[2021-03-09T11:01:44,103][INFO ][o.e.c.c.Coordinator      ] [localhost] cluster UUID [2RPE8JVTQAaFEie6d4A-4w]
[2021-03-09T11:01:44,114][INFO ][o.e.c.c.ClusterBootstrapService] [localhost] no discovery configuration found, will perform best-effort cluster bootstrapping after [3s] unless existing master is discovered
[2021-03-09T11:01:44,220][INFO ][o.e.c.s.MasterService    ] [localhost] elected-as-master ([1] nodes joined)[{localhost}{O29ASbxJQYqCAqMmqXxg3w}{LcwEV0-6THyYT6i7rMHz1w}{127.0.0.1}{127.0.0.1:9300}{cdhilmrstw}{ml.machine_memory=17179869184, xpack.installed=true, transform.node=true, ml.max_open_jobs=20} elect leader, _BECOME_MASTER_TASK_, _FINISH_ELECTION_], term: 3, version: 43, delta: master node changed {previous [], current [{localhost}{O29ASbxJQYqCAqMmqXxg3w}{LcwEV0-6THyYT6i7rMHz1w}{127.0.0.1}{127.0.0.1:9300}{cdhilmrstw}{ml.machine_memory=17179869184, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}]}
[2021-03-09T11:01:44,281][INFO ][o.e.c.s.ClusterApplierService] [localhost] master node changed {previous [], current [{localhost}{O29ASbxJQYqCAqMmqXxg3w}{LcwEV0-6THyYT6i7rMHz1w}{127.0.0.1}{127.0.0.1:9300}{cdhilmrstw}{ml.machine_memory=17179869184, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}]}, term: 3, version: 43, reason: Publication{term=3, version=43}
[2021-03-09T11:01:44,335][INFO ][o.e.h.AbstractHttpServerTransport] [localhost] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}
[2021-03-09T11:01:44,336][INFO ][o.e.n.Node               ] [localhost] started
[2021-03-09T11:01:44,623][INFO ][o.e.l.LicenseService     ] [localhost] license [b3896565-465e-49ec-a104-cd11b9a211ed] mode [basic] - valid
[2021-03-09T11:01:44,624][INFO ][o.e.x.s.s.SecurityStatusChangeListener] [localhost] Active license is now [BASIC]; Security is disabled
[2021-03-09T11:01:44,629][INFO ][o.e.g.GatewayService     ] [localhost] recovered [0] indices into cluster_state

启动成功 浏览器输入 http://localhost:9200/
结果如下

{
  "name" : "localhost",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "2RPE8JVTQAaFEie6d4A-4w",
  "version" : {
    "number" : "7.10.2", // es版本号
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "747e1cc71def077253878a59143c1f785afa92b9",
    "build_date" : "2021-01-13T00:42:12.435326Z",
    "build_snapshot" : false,
    "lucene_version" : "8.7.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

3、安装es插件

常用插件列表

1、x-pack

包括安全（x-pack-security），监视（x-pack-watcher），警报（x-pack-monitoring），报表（x-pack-graph），Elasticsearch SQL（x-pack-sql），跨集群复制（x-pack-ccr）、x-pack-upgrade、x-pack-rollup和机器学习（x-pack-ml）。7.x版本中，es的安全核心功能免费使用。

2、分词插件 ik分词

最流行的分词插件 ik分词

3、pinyin

pinyin插件能够将文档和查询条件在汉字和拼音间转换

4、smartcn

中科院分词器，和ik类似。不同点是smartcn无法自定义词库

5、analysis-icu

Elasticsearch的 ICU 分析器插件 使用 国际化组件 Unicode (ICU) 函数库（详情查看site.project.org）提供丰富的处理 Unicode 工具。 这些包含对处理亚洲语言特别有用的 icu_分词器 ，还有大量对除英语外其他语言进行正确匹配和排序所必须的分词过滤器。

6、ik或smart与pinyin组合使用

es分析器实际上是三个功能的封装，字符过滤器->分词器->token过滤器，按顺序执行 创建自定义分析器，

7、文本解析插件 ingest attachment plugin

这个是基于Apache 文本扩展库Tika插件之上开发的一款适合elasticsearch文本解析插件。在ES5之前使用的是mapper-accachment。ElasticSearch只能处理文本，不能直接处理文档。
Ingest-Attachment是一个开箱即用的插件，替代了较早版本的Mapper-Attachment插件，使用它可以实现对（PDF,DOC,EXCEL等）主流格式文件的文本抽取及自动导入。Elasticsearch5.x新增一个新的特性IngestNode，此功能支持定义命名处理器管道pipeline，pipeline中可以定义多个处理器，在数据插入ElasticSearch之前进行预处理。 而Ingest Attachment Processor Plugin提供了关键的预处理器attachment，支持自动对入库文档的指定字段作为文档文件进行文本抽取。
由于ElasticSearch是基于JSON格式的文档数据库，所以附件文档在插入ElasticSearch之前必须进行Base64编码。

8、SQL支持

1、x-pack-sql
es7.x版本的x-pack自带es sql

9、kibana

es官方提供的可视化套件，集成了es开发工具、报表、仪表盘、数据挖掘等功能https://www.elastic.co/guide/en/kibana/current/targz.html
kibana6.7+才支持汉化

analysis-icu 安装实例

使用命令查看 安装插件列表：bin/elasticsearch-plugin list

安装命令 bin/elasticsearch-plugin install analysis-icu

localhost:elasticsearch-7.10.2 wangyuanyuan$ bin/elasticsearch-plugin install analysis-icu 
future versions of Elasticsearch will require Java 11; your Java version from [/Library/Java/JavaVirtualMachines/jdk1.8.0_201.jdk/Contents/Home/jre] does not meet this requirement
-> Installing analysis-icu
-> Downloading analysis-icu from elastic
[=================================================] 100%   
-> Installed analysis-icu

安装完成 执行 bin/elasticsearch-plugin list

localhost:elasticsearch-7.10.2 wangyuanyuan$ bin/elasticsearch-plugin list
future versions of Elasticsearch will require Java 11; your Java version from [/Library/Java/JavaVirtualMachines/jdk1.8.0_201.jdk/Contents/Home/jre] does not meet this requirement
analysis-icu

重新启动es 执行命令 bin/elasticsearch 浏览器输入 http://localhost:9200/_cat/plugins

结果：

4、安装集群

执行命令：

bin/elasticsearch -E node.name=node0 -E cluster.name=geektime -E path.data=node0_data -d
bin/elasticsearch -E node.name=node1 -E cluster.name=geektime -E path.data=node1_data -d
bin/elasticsearch -E node.name=node2 -E cluster.name=geektime -E path.data=node2_data -d
bin/elasticsearch -E node.name=node3 -E cluster.name=geektime -E path.data=node3_data -d

输入http://localhost:9200/_cat/nodes

结果

127.0.0.1 34 100 15 4.33   cdhilmrstw - node0
127.0.0.1 35 100 15 4.33   cdhilmrstw - node2
127.0.0.1 10 100 15 4.33   cdhilmrstw * node1