Purpose
- Run a cluster on localhost while investigating etcd
- Use a static cluster (So we have no external dependecies for bootstrapping)
Background information
Bootstrap
- Will use static bootstrapping
- Client connection port default is 2379 (Just supporting a single port per node), we will decerement the port for subsequent nodes so we do not get a port conflict
- Peer connection (Raft consensus) port default is 2380 (Just supporting a single port per node), we will increment the port for subsequent nodes so we do not get a port conflict
- Will use /tmp/etcdinv directory for the cluster - If you want the cluster to stick around use a different directory
- If all nodes are stopped and then restarted the cluster will try to restart with this state, if the OS has not already purged this content
- Will write node logs to a file and run process in the background
# etcd bin directory
etcd_bin_dir=/home/pmcgrath/go/src/github.com/coreos/etcd/bin/
# Ensure we have a root directory for the cluster - Note we are using /tmp here, if you want the cluster to stick arounf use a different directory
mkdir -p /tmp/etcdinv
# Run node 1 $etcd_bin_dir./etcd \ -name node1 \ -data-dir /tmp/ectdinv/node1 \
-advertise-client-urls http://localhost:2379 \ -listen-peer-urls http://localhost:2380 \ -listen-client-urls http://localhost:2379 \ -initial-advertise-peer-urls http://localhost:2380 \ -initial-cluster-token MyEtcdCluster \ -initial-cluster node1=http://localhost:2380,node2=http://localhost:2381,node3=http://localhost:2382 \ -initial-cluster-state new &> /tmp/etcdinv/node1.log & # Run node 2 $etcd_bin_dir./etcd \ -name node2 \ -data-dir /tmp/ectdinv/node2 \
-advertise-client-urls http://localhost:2378 \ -listen-peer-urls http://localhost:2381 \ -listen-client-urls http://localhost:2378 \ -initial-advertise-peer-urls http://localhost:2381 \ -initial-cluster-token MyEtcdCluster \ -initial-cluster node1=http://localhost:2380,node2=http://localhost:2381,node3=http://localhost:2382 \ -initial-cluster-state new &> /tmp/etcdinv/node2.log & # Run node 3 $etcd_bin_dir./etcd \ -name node3 \ -data-dir /tmp/ectdinv/node3 \
-advertise-client-urls http://localhost:2377 \ -listen-peer-urls http://localhost:2382 \ -listen-client-urls http://localhost:2377 \ -initial-advertise-peer-urls http://localhost:2382 \ -initial-cluster-token MyEtcdCluster \ -initial-cluster node1=http://localhost:2380,node2=http://localhost:2381,node3=http://localhost:2382 \ -initial-cluster-state new &> /tmp/etcdinv/node3.log & # List nodes ETCDCTL_PEERS=http://127.0.0.1:2379 $etcd_bin_dir/etcdctl member list
- You can see the cluster node pids using
- pidof etcd
- ps aux | grep etcd
Interacting with the cluster using etcdctl
- Will use the client port 2379 based on this
- etcdctl defaults to 4001 at this time
- I could have added an extra client url for 4001 when bring up the nodes, but I'm guessing 4001 will be removed at some stage
# etcd bin directory
etcd_bin_dir=/home/pmcgrath/go/src/github.com/coreos/etcd/bin/
# Using node1
# Write a key ETCDCTL_PEERS=http://127.0.0.1:2379 $etcd_bin_dir/etcdctl set /dir1/key1 value1 # Should echo value1 # Read key ETCDCTL_PEERS=http://127.0.0.1:2379 $etcd_bin_dir/etcdctl get /dir1/key1 # Should echo value1 # Using node3 # Read key ETCDCTL_PEERS=http://127.0.0.1:2377 $etcd_bin_dir/etcdctl get /dir1/key1 # Should echo value1
Kill one of the nodes
# etcd bin directory
etcd_bin_dir=/home/pmcgrath/go/src/github.com/coreos/etcd/bin/
# Kill node2
pidof etcd
# Should only have 3 pids kill $(ps aux | grep 'etcd \-name node2' | cut -d ' ' -f 2) pidof etcd # Should only have 2 pids # Read key using node1 ETCDCTL_PEERS=http://127.0.0.1:2379 $etcd_bin_dir/etcdctl get /dir1/key1 # Should echo value1 # Read key using node2 ETCDCTL_PEERS=http://127.0.0.1:2378 $etcd_bin_dir/etcdctl get /dir1/key1 # Should fail indicating cluster node could not available # Read key using node3 ETCDCTL_PEERS=http://127.0.0.1:2377 $etcd_bin_dir/etcdctl get /dir1/key1 # Should echo value1
Using a proxy
# etcd bin directory
etcd_bin_dir=/home/pmcgrath/go/src/github.com/coreos/etcd/bin/
# Run a read write proxy - on 8080
$etcd_bin_dir/etcd \ -proxy on \ -name proxy \ -listen-client-urls http://localhost:8080 \ -initial-cluster node1=http://localhost:2380,node2=http://localhost:2381,node3=http://localhost:2382 &> /tmp/etcdinv/proxy.log & # Read existing key ETCDCTL_PEERS=http://127.0.0.1:8080 $etcd_bin_dir/etcdctl get /dir1/key1 # Should echo value1 # Write a key ETCDCTL_PEERS=http://127.0.0.1:8080 $etcd_bin_dir/etcdctl set /dir1/key2 value2 # Should echo value2 # Read existing key ETCDCTL_PEERS=http://127.0.0.1:8080 $etcd_bin_dir/etcdctl get /dir1/key1 # Should echo value2
./etcdctl -peers 127.0.0.1:2379 member list