​Nginx K8s + Docker 部署 ，虚拟机部署教程。

Nginx K8s + Docker 部署 ，虚拟机部署教程！

Linux 虚拟机 部署

1、下载nginx:

http://nginx.org/download/

2、选择一个版本进行下载：

wget http://nginx.org/download/nginx-1.14.0.tar.gz

3、解压：

tar -xvf nginx-1.14.0.tar.gz

4、安装依赖包：

yum -y install gcc pcre-devel zlib-devel openssl openssl-devel

5、编译、安装

cd $NGINX_HOME


./configure --prefix=/usr/local/nginx --with-http_stub_status_module --with-http_realip_module


make
make install

6、修改nginx.conf

#user  nobody;
worker_processes  1;
#error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;
#pid        logs/nginx.pid;
events {
    worker_connections  1024;
}


http {
    include       mime.types;
    default_type  application/octet-stream;
    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    #                  '$status $body_bytes_sent "$http_referer" '
    #                  '"$http_user_agent" "$http_x_forwarded_for"';
    #access_log  logs/access.log  main;
    sendfile        on;
    #tcp_nopush     on;
    #keepalive_timeout  0;
    keepalive_timeout  65;
    #gzip  on;
    upstream admin-server {
        server 10.68.60.100:18060    max_fails=2  fail_timeout=30;
        server 10.68.60.101:18060    max_fails=2  fail_timeout=30;
        server 10.68.60.110:18060    max_fails=2  fail_timeout=30;
    }
  
    upstream api-server {
        server 10.68.60.100:18082    max_fails=2  fail_timeout=30;
        server 10.68.60.101:18082    max_fails=2  fail_timeout=30;
        server 10.68.60.110:18082    max_fails=2  fail_timeout=30;
    }


    server {
                listen                  8080;
                server_name             api-server;
                keepalive_timeout       30;
                location / {
                        proxy_pass http://api-server;
                        proxy_set_header X-Forwarded-Host $host;
                        proxy_set_header X-Forwarded-Server $host;
                        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                        proxy_set_header Host $host:$server_port;




                }
    }


    server {
                listen                  8081;
                server_name             admin-server;
                keepalive_timeout       30;
                proxy_set_header X-Forwarded-Host $host;
                proxy_set_header X-Forwarded-Server $host;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header Host $host:$server_port;
                location / {
                        proxy_pass http://admin-server;
                        proxy_set_header X-Forwarded-Host $host;
                        proxy_set_header X-Forwarded-Server $host;
                        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                        proxy_set_header Host $host:$server_port;
                }
    }
}

7、启动nginx:

$NGINX_HOMT/sbin/nginx

8、重启nginx:

$NGINX_HOMT/sbin/nginx -s stop
$NGINX_HOMT/sbin/nginx

9、热更新：

$NGINX_HOMT/sbin/nginx -s reload

Docker + k8s 部署

前提：假定你已经安装并集成好docker和k8s基础环境！

1、创建docker构建目录：

mkdir -p /opt/docker/build/nginx

2、下载nginx：

cd /opt/docker/build/nginx
wget http://nginx.org/download/nginx-1.14.0.tar.gz

3、编写Dockerfile，内容如下：

FROM ansible/centos7-ansible


ADD nginx-1.14.0.tar.gz /


RUN mkdir -p /opt/soft &&\
  mkdir -p /opt/tarball &&\
  mkdir -p /opt/src &&\
  touch /opt/daemon.log &&\
  mv /nginx-1.14.0 /opt/src &&\
  yum -y install gcc pcre-devel zlib-devel openssl openssl-devel &&\
  cd /opt/src/nginx-1.14.0 &&\
  ./configure --prefix=/usr/local/nginx --with-http_stub_status_module --with-http_realip_module &&\
  make &&\
  make install &&\
  cp -R /usr/share/zoneinfo/Asia/Shanghai /etc/localtime




ENTRYPOINT [ "sh", "-c", "/usr/local/nginx/sbin/nginx && tail -f /opt/daemon.log" ]

4、构建docker 镜像：

cd /opt/docker/build/nginx


docker build --network host -t 10.68.60.103:5000/nginx:1.14.0 -f Dockerfile .

注意：10.68.60.103:5000需要根据你自己的docker私服地址修改。

5、推送镜像到私服：

docker push 10.68.60.103:5000/nginx:1.14.0

6、编写k8s nginx-deployment.yaml文件，内容如下：

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  namespace: my-namespace
  labels:
    app: nginx-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx-pod
  template:
    metadata:
      labels:
        app: nginx-pod
    spec:
      nodeSelector:
        deploy.nginx: "true"
      restartPolicy: Always
      containers:
      - name: nginx-container
        image: "10.68.60.103:5000/nginx:1.14.0"
        ports:
#        - containerPort: 80
        volumeMounts:
        - mountPath: "/usr/local/nginx/conf/nginx.conf"
          name: "nginx-volume-conf"
        - mountPath: "/usr/local/nginx/logs"
          name: "nginx-volume-log"
      imagePullSecrets:
      - name: regcred
      volumes:
      - name: "nginx-volume-conf"
        hostPath:
          path: "/opt/apps-mgr/nginx/conf/nginx.conf"
          type: File
      - name: "nginx-volume-log"
        hostPath:
          path: "/opt/apps-mgr/nginx/logs"
          type: DirectoryOrCreate

注意：/opt/apps-mgr/nginx/conf/nginx.conf文件需要提前配置好,笔者这里配置文件如下：

#user  nobody;
worker_processes  1;
#error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;
#pid        logs/nginx.pid;
events {
    worker_connections  1024;
}


http {
    include       mime.types;
    default_type  application/octet-stream;
    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    #                  '$status $body_bytes_sent "$http_referer" '
    #                  '"$http_user_agent" "$http_x_forwarded_for"';
    #access_log  logs/access.log  main;
    sendfile        on;
    #tcp_nopush     on;
    #keepalive_timeout  0;
    keepalive_timeout  65;
    #gzip  on;
    upstream admin-server {
        server 10.68.60.100:18060    max_fails=2  fail_timeout=30;
        server 10.68.60.101:18060    max_fails=2  fail_timeout=30;
        server 10.68.60.110:18060    max_fails=2  fail_timeout=30;
    }
  
    upstream api-server {
        server 10.68.60.100:18082    max_fails=2  fail_timeout=30;
        server 10.68.60.101:18082    max_fails=2  fail_timeout=30;
        server 10.68.60.110:18082    max_fails=2  fail_timeout=30;
    }


    server {
                listen                  8080;
                server_name             api-server;
                keepalive_timeout       30;
                location / {
                        proxy_pass http://api-server;
                        proxy_set_header X-Forwarded-Host $host;
                        proxy_set_header X-Forwarded-Server $host;
                        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                        proxy_set_header Host $host:$server_port;




                }
    }


    server {
                listen                  8081;
                server_name             admin-server;
                keepalive_timeout       30;
                proxy_set_header X-Forwarded-Host $host;
                proxy_set_header X-Forwarded-Server $host;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header Host $host:$server_port;
                location / {
                        proxy_pass http://admin-server;
                        proxy_set_header X-Forwarded-Host $host;
                        proxy_set_header X-Forwarded-Server $host;
                        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                        proxy_set_header Host $host:$server_port;
                }
    }
}

初步可以先用默认的配置，等启动成功后再修改，修改后需要进入Pod重启Nginx，进入命令如下：

kubectl get pods -A | grep nginx
kubectl exec -it -n my-namespace nginx-deployment-5dcf489c44-f9c4r
/usr/local/nginx/sbin/nginx -s stop && /usr/local/nginx/sbin/nginx

另外nodeSelector部分作用是指定nginx运行在某个节点上。需要提前通过下面命令给指定节点打上标签，例如指定允许在node2时：

kubectl label nodes node2 deploy.nginx=true

7、执行deployment：

kubectl apply -f nginx-deployment.yaml
kubectl get pods -A
kubectl -n my-namespace describe pod xxxxx
kubectl -n my-namespace get events

8、编写k8s nginx-service.yaml文件，内容如下：

apiVersion: v1
kind: Service
metadata:
  namespace: my-namespace
  name: nginx-api-server-service
spec:
  type: NodePort
  selector:
    app: nginx-pod
  ports:
    - protocol: TCP
      port: 8080
      targetPort: 8080
      nodePort: 8080


---


apiVersion: v1
kind: Service
metadata:
  namespace: my-namespace
  name: nginx-admin-server-service
spec:
  type: NodePort
  selector:
    app: nginx-pod
  ports:
    - protocol: TCP
      port: 8081
      targetPort: 8081
      nodePort: 8081

9、执行service：

kubectl apply -f nginx-service.yaml
kubectl get service -A
kubectl -n my-namespace describe service xxxxx
kubectl -n my-namespace get events

---------- 正文结束 ----------

长按扫码关注微信公众号

Java软件编程之家