03.德国博士练习_02_admin_cluster

1. exercise: allocation filter

# ** EXAM OBJECTIVE: CLUSTER ADMINISTRATION **
# GOAL: Allocate the shards in a way that satisfies a given set of
# requirements
# REQUIRED SETUP: /

# Download the exam version of Elasticsearch
# Deploy the cluster `eoc-06-cluster`, with three nodes named
# `node1`, `node2`, and `node3`
# Configure the Zen Discovery module of each node so that they can
# communicate
# Start the cluster

# Create the index `hamlet-1` with two primary shards and one   replica

# Add some documents to `hamlet-1` by running the command below 
PUT hamlet-1/_doc/_bulk
{"index":{"_index":"hamlet-1","_id":0}}
{"line_number":"1","speaker":"BERNARDO","text_entry":"Whos there?"}
{"index":{"_index":"hamlet-1","_id":1}}
{"line_number":"2","speaker":"FRANCISCO","text_entry":"Nay, answer me:
stand, and unfold yourself."}
{"index":{"_index":"hamlet-1","_id":2}}
{"line_number":"3","speaker":"BERNARDO","text_entry":"Long live the
king!"}
{"index":{"_index":"hamlet-1","_id":3}}
{"line_number":"4","speaker":"FRANCISCO","text_entry":"Bernardo?"}
{"index":{"_index":"hamlet-1","_id":4}}
{"line_number":"5","speaker":"BERNARDO","text_entry":"He."}



# Create the index `hamlet-2` with two primary shard and one replica
# Add some documents to `hamlet-2` by running the command belowPUT

hamlet-2/_doc/_bulk
{"index":{"_index":"hamlet-2","_id":5}}
{"line_number":"6","speaker":"FRANCISCO","text_entry":"You come most carefully upon your hour."}
{"index":{"_index":"hamlet-2","_id":6}}
{"line_number":"7","speaker":"BERNARDO","text_entry":"Tis now struck
twelve; get thee to bed, Francisco."}
{"index":{"_index":"hamlet-2","_id":7}}
{"line_number":"8","speaker":"FRANCISCO","text_entry":"For this relief
much thanks: tis bitter cold,"}
{"index":{"_index":"hamlet-2","_id":8}}
{"line_number":"9","speaker":"FRANCISCO","text_entry":"And I am sick at
heart."}
{"index":{"_index":"hamlet-2","_id":9}}
{"line_number":"10","speaker":"BERNARDO","text_entry":"Have you had quiet
guard?"}

PUT hamlet-1
{
  "settings": {
    "number_of_replicas": 1,
    "number_of_shards": 2
  }
}


PUT hamlet-2
{
  "settings": {
    "number_of_replicas": 1,
    "number_of_shards": 2
  }
}

验证

# Check that the replicas of indices `hamlet-1` and `hamlet-2` have
# been allocated
# Check the distribution of primary shards and replicas of indices
# `hamlet-1` and `hamlet-2` across the nodes of the cluster


GET _cat/shards/hamlet-1?v
GET _cat/shards/hamlet-2?v

# Configure `hamlet-1` to allocate both primary shards to `node2`,
# using the node name
# Verify the success of the last action by using the _cat API
# Configure `hamlet-2` so that no primary shard is allocated to
# `node3`
# Verify the success of the last action by using the _cat API
# Remove any allocation filter setting associated with `hamlet-1`
# and `hamlet-2`

PUT hamlet-1/_settings
{
  "number_of_replicas": 0
}

这里一开始眼残，把_name看成 _node了，还是手写了，整了半天没有成，哎，千万别犯这种低级错误啊，要相信官方，多去排查自己到底哪里出错了。
认真仔细，可能是一个很简单的错误，耽误了你，这就是应试的心态，注意调整

PUT hamlet-1/_settings
{
  "index.routing.allocation.include._name":"node3"
}

PUT hamlet-2/_settings
{
  "index.routing.allocation.exclude._name":"node3"
}



恢复

PUT hamlet-1/_settings
{
  "index.routing.allocation.include._name":null
}

PUT hamlet-2/_settings
{
  "index.routing.allocation.exclude._name":null
}

force awaness

# Let's assume that we deployed the `eoc-06-cluster` cluster across
# two availability zones, named `earth` and `mars`. Add the
# attribute `AZ` to the nodes configuration, and set its value
# to "earth" for `node1` and `node2`, and to "mars" for `node3`
# Restart the cluster
# Configure the cluster to force shard allocation awareness based on
# the two availability zones, and persist such configuration
# across cluster restarts
# Verify the success of the last action by using the _cat APIo




PUT _cluster/settings
{
  "persistent": {
    "cluster.routing.allocation.awareness.attributes":"AZ",
    "cluster.routing.allocation.awareness.force.AZ.values": ["earth","mars"]
  }
}

index-filter

# Configure the cluster to reflect a hot/warm architecture, with
# `node1` as the only hot node
# Configure the `hamlet-1` index to allocate its shards only to warm
# nodes
# Verify the success of the last action by using the _cat API \
# Remove the hot/warm shard filtering configuration from the
# `hamlet-1` configuration


# Let's assume that the nodes have either a "large" or "small" local
# storage. Add the attribute `storage` to the nodes config, and
# set its value so that `node2` is the only with a "small"
# storage
# Configure the `hamlet-2` index to allocate its shards only to
# nodes with a large storage size
# Verify the success of the last action by using the _cat API

当时把node写成onde了，也是服了我自己了，晕，搞了半天，千万不能急，而且尽可能从doc上复制

node.attr.tempreture: hot
node.attr.tempreture: warm  
node.attr.tempreture: warm  

重启



PUT hamlet-1/_settings
{
  "index.routing.allocation.include.tempreture":"warm"
}

GET _cat/shards/hamlet-1

node.attr.storage: large
node.attr.storage: small
node.attr.storage: large

重启


PUT hamlet-2/_settings
{
  "index.routing.allocation.include.storage":"large"
}


GET _cat/shards/hamlet-2

2. exercise02: backup and cross-cluster search

# ** EXAM OBJECTIVE: CLUSTER ADMINISTRATION **
# GOAL: Backup and cross-cluster search

# REQUIRED SETUP: /
Let’s create a one-node cluster and index some data in it.
# Download the exam version of Elasticsearch
# Deploy the cluster `eoc-06-original-cluster`, with one node named
# `node-1`
# Start the cluster
# Create the index `hamlet` and add some documents by running the
# following _bulk command

PUT hamlet/_doc/_bulk
{"index":{"_index":"hamlet","_id":0}}
{"line_number":"1","speaker":"BERNARDO","text_entry":"Whos there?"}
{"index":{"_index":"hamlet","_id":1}}
{"line_number":"2","speaker":"FRANCISCO","text_entry":"Nay, answer me:
stand, and unfold yourself."}
{"index":{"_index":"hamlet","_id":2}}
{"line_number":"3","speaker":"BERNARDO","text_entry":"Long live the
king!"}
{"index":{"_index":"hamlet","_id":3}}
{"line_number":"4","speaker":"FRANCISCO","text_entry":"Bernardo?"}
{"index":{"_index":"hamlet","_id":4}}
{"line_number":"5","speaker":"BERNARDO","text_entry":"He."}

题目一

# Configure `node-1` to support a shared file system repository for
# backups located in
# (i) "[home_folder]/repo" and
# (ii) "[home_folder]/elastic/repo" - e.g., "glc/elastic/repo"
# Create the `hamlet_backup` shared file system repository in
# "[home_folder]/elastic/repo"
# Create a snapshot of the `hamlet` index, so that the snapshot
# (i) is named `hamlet_snapshot_1`,
# (ii) is stored into `hamlet_backup`
# Delete the index `hamlet`
# Restore the index `hamlet` using `hamlet_snapshot_1`

题解

mkdir repo
mkdir -p elastic/repo

yml中设置
path.repo: ["/home/deploy/search/log-manager/germany/single-node01/elasticsearch-7.2.0/repo","/home/deploy/search/log-manager/germany/single-node01/elasticsearch-7.2.0/elastic/repo"]


下面均为kibana操作

PUT _snapshot/hamlet_backup
{
  "type": "fs",
  "settings": {
    "location": "/home/deploy/search/log-manager/germany/single-node01/elasticsearch-7.2.0/elastic/repo"
  }
}

PUT _snapshot/hamlet_backup/hamlet_snapshot_1
{
  "indices": "hamlet"
}

GET _cat/indices/*

GET _snapshot/hamlet_backup/*

DELETE hamlet

POST _snapshot/hamlet_backup/hamlet_snapshot_1/_restore

GET hamlet/_count

题目二

# Deploy a second cluster `eoc-06-adaptation-cluster`, with one node
# named `node-2`
# Start the cluster
# Create the index `hamlet-pirate` on `node-2` and add documents
# using the _bulk command
PUT hamlet-pirate/_doc/_bulk
{"index":{"_index":"hamlet-pirate","_id":5}}
{"line_number":"6","speaker":"FRANCISCO","text_entry":"Ahoy Matey! Ye
come most carefully upon yer hour."}
{"index":{"_index":"hamlet-pirate","_id":6}}
{"line_number":"7","speaker":"BERNARDO","text_entry":"Aye! Tis now struck
twelve; get ye to bed, Francisco."}
{"index":{"_index":"hamlet-pirate","_id":7}}
{"line_number":"8","speaker":"FRANCISCO","text_entry":"For this relief
much thanks, son of a biscuit eater"}
{"index":{"_index":"hamlet-pirate","_id":8}}
{"line_number":"9","speaker":"BERNARDO","text_entry":"Arrrrrrrrh!"}


# Enable cross cluster search on `eoc-06-adaptation-cluster`, so
# that
# (i) the name of the remote cluster is `original`,
# (ii) the seed is `node-1`, which is listening on the default
# transport port,
# (iii) the cross cluster configuration persists across multiple
# restarts
# Run the cross-cluster query below to check your setup

GET /original:hamlet,hamlet-pirate/_search
{
 "query": {
 "match": {
 "speaker": "BERNARDO"
 }
 }
}

配置的时候最开始把端口写错了，需要注意

PUT _cluster/settings
{
  "persistent": {
    "cluster": {
      "remote": {
        "original": {
          "seeds": [
            "10.76.3.145:20300"
          ],
          "transport.ping_schedule": "30s"
        }
       
      }
    }
  }
}



GET original:hamlet,hamlet-pirate/_search
{
  "query": {
    "match": {
      "speaker": "BERNARDO"
    }
  }
}