准备
在RHEL 7上部署Kubernetes之前, 需要准备如下资源:
- 安装Git;
- Docker的YUM源;
- 编译Kubernetes;
- 编译Etcd;
- 3台VM Hosts, 10.224.106.127(docker01), 10.224.106.128/130(docker02,docker03)
安装步骤
安装Git及编译Kubernetes和Etcd
docker01安装kubernetes apiserver, controller-manager, kubecfg和etcd, docker02/03安装docker和kubernetes kubelet和proxy.
安装Git
yum install git
编译Kubernetes和Etcd
cd /tmp
git clone https://github.com/GoogleCloudPlatform/kubernetes.git
cd kubernetes/hack
./build-go.sh
cd /tmp
git clone https://github.com/coreos/etcd.git
cd etcd
./build
将编译后Kubernetes的apiserver, controller-manager和kubecfg移到/opt/kubernetes/bin目录下.
mkdir -p /opt/kubernetes/bin
cd /tmp/kubernetes/output/go/bin
cp apiserver /opt/kubernetes/bin
cp controller-manager /opt/kubernetes/bin
cp kubecfg /opt/kubernetes/bin
cd /tmp/etcd/bin
cp etcd /opt/kubernetes/bin
配置apiserver, controller-manager, etcd的unit file, 首先做如下操作:
cd /etc/systemd/system/
touch kubernetes-apiserver.service
touch kubernetes-controller-manager.service
touch etcd.service
以下是kubernetes-apiserver.service, kubernetes-controller-manager.service, etcd.service的unit file配置:
#kubernetes-apiserver.service
[Unit]
Description=Kubernetes API Server
[Service]
ExecStart=/opt/kubernetes/bin/apiserver \
--address=10.224.106.127 \
--port=8080 \
--etcd_servers=http://10.224.106.127:4001 \
--machines=10.224.106.128,10.224.106.130 \
--logtostderr=true
[Install]
WantedBy=multi-user.target
#kubernetes-controller-manager.service
[Unit]
Description=Kubernetes Controller Manager
[Service]
ExecStart=/opt/kubernetes/bin/controller-manager \
--etcd_servers=http://10.224.106.127:4001 \
--master=10.224.106.127:8080 \
--logtostderr
[Install]
WantedBy=multi-user.target
#etcd.service
[Unit]
Description=Etcd Server
After=network.target
[Service]
Type=simple
# etc logs to the journal directly, suppress double logging
StandardOutput=null
WorkingDirectory=/var/lib/etcd
ExecStart=/opt/kubernetes/bin/etcd -addr 10.224.106.127:4001 -peers 10.224.106.127:7001
[Install]
WantedBy=multi-user.target
启动apiserver, controller-manager, etcd服务,
systemctl daemon-reload
systemctl enable kubernetes-apiserver
systemctl enable kubernetes-controller-manager
systemctl enable etcd
systemctl start kubernetes-apiserver
systemctl start kubernetes-controller-manager
systemctl start etcd
检查apiserver, controller-manager, etcd服务是否启动以及8080, 4001,7001 port是否打开,
[root@docker01 system]# ps -ef | grep -i kubernetes
root 12317 1 0 Aug15 ? 00:05:28 /opt/kubernetes/bin/apiserver --address=10.224.106.127 --port=8080 --etcd_servers=http://10.224.106.127:4001 --machines=10.224.106.128,10.224.106.130 --logtostderr=true
root 12325 1 0 Aug15 ? 00:02:00 /opt/kubernetes/bin/controller-manager --etcd_servers=http://10.224.106.127:4001 --master=10.224.106.127:8080 --logtostderr
root 12333 1 0 Aug15 ? 00:02:39 /opt/kubernetes/bin/etcd -addr 10.224.106.127:4001 -peers 10.224.106.127:7001
telnet 8080, 4001, 7001port,
telnet 10.224.106.127 8080
telnet 10.224.106.127 4001
telnet 10.224.106.127 7001
iptables -I INPUT -p tcp --dport 4001 -j ACCEPT
iptables -I INPUT -p tcp --dport 7001 -j ACCEPT
iptables -I INPUT -p tcp --dport 8080 -j ACCEPT
配置Docker的YUM源和安装Docker
配置YUM源:
cd /tmp
wget http://dl.fedoraproject.org/pub/epel/beta/7/x86_64/epel-release-7-0.2.noarch.rpm
ls *.rpm
yum install epel-release-7-0.2.noarch.rpm
安装Docker
yum install docker-io
systemctl enable docker
systemctl start docker
执行
docker version验证Docker是否安装成功, 如果安装成功, 输出如下信息:
[root@docker02 system]# docker version
Client version: 1.0.0
Client API version: 1.12
Go version (client): go1.2.2
Git commit (client): 63fe64c/1.0.0
Server version: 1.0.0
Server API version: 1.12
Go version (server): go1.2.2
Git commit (server): 63fe64c/1.0.0
安装kubelet, proxy服务
同docker01一样, 在docker02和docker03上创建/opt/kubernetes/bin,mkdir -p /opt/kubernetes/bin
将在docker01上编译的kubelet和proxy从/tmp/kubernetes/output/go/bin复制到/opt/kubernetes/bin目录下, 然后给kubelet和proxy配置unit file信息.
cd /etc/systemd/system/
touch kubernetes-kubelet.service
touch kubernetes-proxy.service
以下是docker02上kubelet和proxy的具体unit file配置信息:
<pre name="code" class="plain">#kubernetes-kubelet.service
[Unit]
Description=Kubernetes Kubelet
[Service]
ExecStart=/opt/kubernetes/bin/kubelet \
--logtostderr=true \
--etcd_servers=http://10.224.106.127:4001 \
--address=10.224.106.128 \
--port=10250 \
--hostname_override=10.224.106.128
[Install]
WantedBy=multi-user.target
#kubernetes-proxy.service
[Unit]
Description=Kubernetes Proxy
[Service]
ExecStart=/opt/kubernetes/bin/proxy \
--logtostderr=true \
--etcd_servers=http://10.224.106.127:4001
[Install]
WantedBy=multi-user.target
docker03上kubelet和proxy的具体unit file配置信息:
#kubernetes-kubelet.service
[Unit]
Description=Kubernetes Kubelet
[Service]
ExecStart=/opt/kubernetes/bin/kubelet \
--logtostderr=true \
--etcd_servers=http://10.224.106.127:4001 \
--address=10.224.106.130 \
--port=10250 \
--hostname_override=10.224.106.130
[Install]
WantedBy=multi-user.target
#kubernetes-proxy.service
[Unit]
Description=Kubernetes Proxy
[Service]
ExecStart=/opt/kubernetes/bin/proxy \
--logtostderr=true \
--etcd_servers=http://10.224.106.127:4001
[Install]
WantedBy=multi-user.target
启动kubelet和proxy服务:
systemctl daemon-reload
systemctl enable kubernetes-kubelet
systemctl enable kubernetes-proxy
systemctl start kubernetes-kubelet
systemctl start kubernetes-proxy
[root@docker02 system]# ps -ef | grep kubernetes
root 3676 1 0 Aug15 ? 00:01:06 /opt/kubernetes/bin/kubelet --logtostderr=true --etcd_servers=http://10.224.106.127:4001 --address=10.224.106.128 --port=10250 --hostname_override=10.224.106.128
root 5116 1 0 00:32 ? 00:00:00 /opt/kubernetes/bin/proxy --logtostderr=true --etcd_servers=http://10.224.106.127:4001
同dock01一样,
telent docker02/03的10250 port是否打开, 如果没有, 可能需要检查iptables,
iptables -I INPUT -p tcp --dport 10250 -j ACCEPT
至此, kubernetes, docker在docker01, docker02, docker03已安装成功, 接下来我们使用kubernetes来创建Pod, Service, replicationController.
Examples
创建Redis Master Pod
在docker01上, 我们cd到/tmp/kubernetes/example/gestbook(以下所有关于创建Pod, Service, replicationController都是在这个目录下执行的), 然后执行如下命令创建Pod:
[root@docker01 guestbook]# /opt/kubernetes/bin/kubecfg -h http://10.224.106.127:8080 -c redis-master.json create pods
Name Image(s) Host Labels
---------- ---------- ---------- ----------
redis-master-2 dockerfile/redis / name=redis-master
[root@docker01 guestbook]#
[root@docker01 guestbook]#
[root@docker01 guestbook]# /opt/kubernetes/bin/kubecfg -h http://10.224.106.127:8080 list pods
Name Image(s) Host Labels
---------- ---------- ---------- ----------
redis-master-2 dockerfile/redis 10.224.106.128/ name=redis-master
根据上面的输出可知名为redis-master-2的Pod被创建在10.224.106.128(docker02), 在docke02上执行
docker ps命令会有如下输出, 其中一个是redis container, 另一个是network container, network container给redis container提供网络, redis container的服务端口都通过network container暴露给外界, 因此每创建一个Pod, 就会相应的创建一个network container.
[root@docker02 system]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
28fba4be5e72 dockerfile/redis:latest redis-server /etc/re 4 minutes ago Up 4 minutes k8s--master--redis_-_master_-_2.etcd--be3edd4b
f8355960c774 kubernetes/pause:latest /pause 4 minutes ago Up 4 minutes 0.0.0.0:6379->6379/tcp k8s--net--redis_-_master_-_2.etcd--bdcb4fa0
创建Redis Master Services
执行如下命令创建Services:
[root@docker01 guestbook]# /opt/kubernetes/bin/kubecfg -h http://10.224.106.127:8080 -c redis-master-service.json create services
Name Labels Selector Port
---------- ---------- ---------- ----------
redismaster name=redis-master 10000
[root@docker01 guestbook]#
[root@docker01 guestbook]#
[root@docker01 guestbook]# /opt/kubernetes/bin/kubecfg -h http://10.224.106.127:8080 list services
Name Labels Selector Port
---------- ---------- ---------- ----------
redismaster name=redis-master 10000
执行这个命令之后, 你能在docker02, docker03看到有一个10000 port被监听, 标志着Redis Master services启动成功.
[root@docker02 system]# netstat -anpl | grep 10000
tcp6 0 0 :::10000 :::* LISTEN 5385/proxy
[root@docker03 ~]# netstat -anpl | grep 10000
tcp6 0 0 :::10000 :::* LISTEN 5986/proxy
创建Replicated Slave Pods
执行如下命令创建Replicated Slave Pods:
[root@docker01 guestbook]# /opt/kubernetes/bin/kubecfg -h http://10.224.106.127:8080 -c redis-slave-controller.json create replicationControllers
I0817 01:34:12.358656 13986 request.go:249] Waiting for completion of /operations/192
Name Image(s) Selector Replicas
---------- ---------- ---------- ----------
redisSlaveController brendanburns/redis-slave name=redisslave 2
我们发现Replicas是2, 这表明根据redis-slave-controller.json的pod template会创建2个pod, 执行如下命令可知其中一个pod被创建在10.224.106.128, 而另一个pod被创建在10.224.106.130上, 这是因为Kubernetes采用Round Robin Schedule算法来选择host, 所以这2个pod分布在不同的host上, 而且在docker02和docker03上分别建立了一个container, 具体看输出信息.
[root@docker01 guestbook]# /opt/kubernetes/bin/kubecfg -h http://10.224.106.127:8080 list pods
Name Image(s) Host Labels
---------- ---------- ---------- ----------
redis-master-2 dockerfile/redis 10.224.106.128/ name=redis-master
22efb4af-25d0-11e4-a43e-005056b47e1e brendanburns/redis-slave <span style="background-color: rgb(255, 255, 102);">10.224.106.128/</span> name=redisslave,replicationController=redisSlaveController
22efde60-25d0-11e4-a43e-005056b47e1e brendanburns/redis-slave <span style="background-color: rgb(102, 102, 204);">10.224.106.130/</span> name=redisslave,replicationController=redisSlaveController
[root@docker02 system]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
21e08fcdb6e2 brendanburns/redis-slave:latest /bin/sh -c /run.sh 7 minutes ago Up 7 minutes k8s--slave--22efb4af_-_25d0_-_11e4_-_a43e_-_005056b47e1e.etcd--ac5522f9
bda4f875e5b5 kubernetes/pause:latest /pause 7 minutes ago Up 7 minutes 0.0.0.0:6380->6379/tcp k8s--net--22efb4af_-_25d0_-_11e4_-_a43e_-_005056b47e1e.etcd--eb7a8b2a
[root@docker03 ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6b6e87b7b0c6 brendanburns/redis-slave:latest /bin/sh -c /run.sh 7 minutes ago Up 7 minutes k8s--slave--22efde60_-_25d0_-_11e4_-_a43e_-_005056b47e1e.etcd--bf57ab92
ce2d93576d11 kubernetes/pause:latest /pause 7 minutes ago Up 7 minutes 0.0.0.0:6380->6379/tcp k8s--net--22efde60_-_25d0_-_11e4_-_a43e_-_005056b47e1e.etcd--7df88eaf
创建Redis Slave Services
在创建完Slave Pods后, 接着创建Redis Slave Services, 执行如下命令:
[root@docker01 guestbook]# /opt/kubernetes/bin/kubecfg -h http://10.224.106.127:8080 -c redis-slave-service.json create services
Name Labels Selector Port
---------- ---------- ---------- ----------
redisslave name=redisslave name=redisslave 10001
同Redis Master Services一样, Redis Slave Services也会在docker02和docker03上监听10001 port, 具体如下:
[root@docker02 system]# netstat -anpl | grep 10001
tcp6 0 0 :::10001 :::* LISTEN 5385/proxy
[root@docker03 ~]# netstat -anpl | grep 10001
tcp6 0 0 :::10001 :::* LISTEN 5986/proxy
对于其它的操作, 比如list/delete pods, list/delete services等就不一一举例了, 具体操作可参考kubecfg的官方文档或者联系我. 如果文中有问题或错误, 欢迎大家提出以便修改.
参考资料
- http://developerblog.redhat.com/2014/07/31/getting-started-with-kubernetes-docker-on-fedora/
- https://github.com/GoogleCloudPlatform/kubernetes/blob/master/examples/guestbook/README.md