目录
简介：
容器技术除了docker之外，还有其它项目比如docker开源的containerd，以及之前coreOs的rkt，redhat的podman等等，包括buildkitd，它也是从docker公司开源出来的构建镜像的工具
为了保证容器生态的标准，众多云厂商成立了个组织叫OCI（open container），目的就是制定开放的标准容器规范，目前发布了两个规范，分别是容器运行时runtime spec和image format spec
因此只要遵循OCI的运行时标准和构建镜像标准，无论使用那种镜像构建工具构建的镜像，都可以运行在各个容器运行时上
docker功能比较强大，直接具备构建镜像的功能，但是buildkitd不能直接构建镜像，需要依赖containerd
buildkitd由两部分组成
1、buildkitd(服务端),目前支持runc和containerd作为镜像构建环境，默认是runc,可调整为containerd
2、kuildctl(客户端)，负责解析Dockerfile文件，并向buildkitd发出构建请求
一、部署buildkitd
注：（部署的主机环境要干净，不要同时有docker和containerd两个运行时）
1、下载二进制包
1.1、进入官网GitHub: Let’s build from here · GitHub后搜索buildkit，然后点击moby/buildkit
1.2、点击后进入右边的版本库
1.3、选择一个版本下载amd的二进制包
2、 解压二进制包，并移到全局变量中
下载的二进制包
root@master1:~# ls
buildkit-v0.10.6.linux-amd64.tar.gz snap
解压到全局变量路径下
root@master1:~# tar xvf buildkit-v0.10.6.linux-amd64.tar.gz -C /usr/local/bin/
bin/ #二进制文件目录
bin/buildctl
bin/buildkit-qemu-aarch64
bin/buildkit-qemu-arm
bin/buildkit-qemu-i386
bin/buildkit-qemu-mips64
bin/buildkit-qemu-mips64el
bin/buildkit-qemu-ppc64le
bin/buildkit-qemu-riscv64
bin/buildkit-qemu-s390x
bin/buildkit-runc
bin/buildkitd
二进制文件在解压的bin目录下
root@master1:~# ls /usr/local/bin/bin/
buildctl buildkit-qemu-aarch64 buildkit-qemu-i386 buildkit-qemu-mips64el buildkit-qemu-riscv64 buildkit-runc
buildkitd buildkit-qemu-arm buildkit-qemu-mips64 buildkit-qemu-ppc64le buildkit-qemu-s390x
把bin下的两个命令移到/usr/local/bin下
root@master1:~# mv /usr/local/bin/bin/buildctl /usr/local/bin/buildctl
root@master1:~# mv /usr/local/bin/bin/buildkitd /usr/local/bin/buildkitd
查看下两个命令都移动/usr/local/bin下了，可全局执行了
root@master1:~# ls /usr/local/bin/ | grep build
buildctl
buildkitd
3、创建socket文件，service文件
root@master1:~# vim /lib/systemd/system/buildkit.socket
[Unit]
Description=BuildKit
Documentation=https://github.com/moby/buildkit
[Socket]
ListenStream=%t/buildkit/buildkitd.sock
[Install]
WantedBy=sockets.target
root@master1:~# vim /lib/systemd/system/buildkitd.service
[Unit]
Description=BuildKit
Requires=buildkit.socket
After=buildkit.socketDocumentation=https://github.com/moby/buildkit
[Service]
ExecStart=/usr/local/bin/buildkitd --oci-worker=false --containerd-worker=true
[Install]
WantedBy=multi-user.target
注解：ExecStart=/usr/local/bin/buildkitd --oci-worker=false --containerd-worker=true buildkitd启动方式直接执行二进制文件，且使用containerd构建镜像
4、启动服务
root@master1:~# systemctl daemon-reload
root@master1:~# systemctl start buildkitd
root@master1:~# systemctl enable buildkitd
root@master1:~# systemctl status buildkitd
二、使用nerdctl命令测试镜像构建
1、部署nerdctl客户端工具
官网 https://github.com/containerd/nerdctl/releases
# wget https://github.com/containerd/nerdctl/releases/download/v0.18.0/nerdctl-0.18.0-linux-amd64.tar.gz
# tar xvf nerdctl-0.18.0-linux-amd64.tar.gz
# cp nerdctl /usr/local/bin/
现在登录harbor服务器会报证书错误
2、签发证书，客户端登录harbor
之前 https://blog.csdn.net/weixin_46476452/article/details/127732870 博客中也部署过签发给docker运行时证书，不过这里是containerd运行时，签发的目录有所不同
1、客户端创建签发证书目录
root@master1:~# mkdir /etc/containerd/certs.d/harbor.magedu.net -p
2、harbor服务端进行证书签发（需要提前把crt证书转换成cert格式的证书）
root@harbor:/apps/harbor/certs# openssl x509 -inform PEM -in magedu.net.crt -out magedu.net.cert
root@harbor:/apps/harbor/certs# ls
ca.crt ca.key ca.srl magedu.net.cert magedu.net.crt magedu.net.csr magedu.net.key v3.ext
3、把harbor主机的ca公钥，harbor的cert公钥，harbor的私钥拷贝到客户端
root@harbor:/apps/harbor/certs# scp ca.crt magedu.net.cert magedu.net.key master1:/etc/containerd/certs.d/harbor.magedu.net/
4、客户端查验
root@master1:~# ls /etc/containerd/certs.d/harbor.magedu.net
ca.crt magedu.net.cert magedu.net.key
3、 客户端正常登录harbor服务端
三、制作镜像测试
1、构建基础环境镜像
root@master1:~/ubuntu# vim Dockerfile
FROM ubuntu:22.04
MAINTAINER "zhaoyang 2569220198@qq.com"
#ADD sources.list /etc/apt/sources.list
RUN apt update && apt install -y iproute2 ntpdate tcpdump telnet traceroute nfs-kernel-server nfs-common lrzsz tree openssl libssl-dev libpcre3 libpcre3-dev zlib1g-dev ntpdate tcpdump telnet traceroute gcc openssh-server lrzsz tree openssl libssl-dev libpcre3 libpcre3-dev zlib1g-dev ntpdate tcpdump telnet traceroute iotop unzip zip make
ADD nginx-1.22.0.tar.gz /usr/local/src/
RUN cd /usr/local/src/nginx-1.22.0 && ./configure --prefix=/apps/nginx && make && make install && ln -sv /apps/nginx/sbin/nginx /usr/bin
RUN groupadd -g 2088 nginx && useradd -g nginx -s /usr/sbin/nologin -u 2088 nginx && chown -R nginx.nginx /apps/nginx
ADD nginx.conf /apps/nginx/conf/
ADD frontend.tar.gz /apps/nginx/html/
EXPOSE 80 443
#ENTRYPOINT ["nginx"]
CMD ["nginx","-g","daemon off;"]
构建镜像
nerdctl build -t harbor.magedu.net/magedu/nginx-base:1.22.0 .
上传到harbor
nerdctl push harbor.magedu.net/magedu/nginx-base:1.22.0
2、修改nerdctl的namespace
在没有修改nerdctl的namespce时是看不见k8s的镜像的
创建nerdctl配置文件 root@master1:~# mkdir /etc/nerdctl root@master1:~# vim /etc/nerdctl/nerdctl.toml namespace = "k8s.io" #让nerdctl是使用k8s.io的namespace # k8s默认使用k8s.io namespace, nerdctl默认使用default namspace,因此需要切换至k8s.io的namespace才能显示对方的镜像，tag等修改完nerdctl的namespace后就可以看见k8s的镜像了，说明上面的配置文件生效了
3、 安装cni
官网下载cni https://github.com/containernetworking/plugins/releases
root@master1:~# wget https://github.com/containernetworking/plugins/releases/download/v1.1.1/cni-plugins-linux-amd64-v1.1.1.tgz
root@master1:~# mkdir /opt/cni/bin/ -p
root@master1:~# tar xf cni-plugins-linux-amd64-v1.1.1.tgz -C /opt/cni/bin/
4、构建服务的基础父镜像centos
1、编写dockerfile root@master1:/opt/k8s-data/dockerfile/system/centos# vim Dockerfile #自定义Centos 基础镜像 FROM centos:7.9.2009 MAINTAINER zhaoyang 2569220198@qq.com ADD filebeat-7.12.1-x86_64.rpm /tmp RUN yum install -y /tmp/filebeat-7.12.1-x86_64.rpm vim wget tree lrzsz gcc gcc-c++ automake pcre pcre-devel zlib zlib-devel openssl openssl-devel iproute net-tools iotop && rm -rf /etc/localtime /tmp/filebeat-7.12.1-x86_64.rpm && ln -snf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && useradd nginx -u 2088 2、构建镜像，上传镜像到harbor root@master1:/opt/k8s-data/dockerfile/system/centos# vim build-command.sh #!/bin/bash /usr/local/bin/nerdctl build -t harbor.magedu.net/baseimages/magedu-centos-base:7.9.2009 . /usr/local/bin/nerdctl push harbor.magedu.net/baseimages/magedu-centos-base:7.9.2009查看harbor仓库
5、基于父镜像centos 制作jdk的服务镜像
查看dockerfile同级所需文件 root@master1:/opt/k8s-data/dockerfile/web/pub-images/jdk-1.8.212# ls Dockerfile jdk-8u212-linux-x64.tar.gz profile root@master1:/opt/k8s-data/dockerfile/web/pub-images/jdk-1.8.212# cat profile pathmunge () { case ":${PATH}:" in *:"$1":*) ;; *) if [ "$2" = "after" ] ; then PATH=$PATH:$1 else PATH=$1:$PATH fi esac } if [ -x /usr/bin/id ]; then if [ -z "$EUID" ]; then # ksh workaround EUID=`/usr/bin/id -u` UID=`/usr/bin/id -ru` fi USER="`/usr/bin/id -un`" LOGNAME=$USER MAIL="/var/spool/mail/$USER" fi # Path manipulation if [ "$EUID" = "0" ]; then pathmunge /usr/sbin pathmunge /usr/local/sbin else pathmunge /usr/local/sbin after pathmunge /usr/sbin after fi HOSTNAME=`/usr/bin/hostname 2>/dev/null` HISTSIZE=1000 if [ "$HISTCONTROL" = "ignorespace" ] ; then export HISTCONTROL=ignoreboth else export HISTCONTROL=ignoredups fi export PATH USER LOGNAME MAIL HOSTNAME HISTSIZE HISTCONTROL # By default, we want umask to get set. This sets it for login shell # Current threshold for system reserved uid/gids is 200 # You could check uidgid reservation validity in # /usr/share/doc/setup-*/uidgid file if [ $UID -gt 199 ] && [ "`/usr/bin/id -gn`" = "`/usr/bin/id -un`" ]; then umask 002 else umask 022 fi for i in /etc/profile.d/*.sh /etc/profile.d/sh.local ; do if [ -r "$i" ]; then if [ "${-#*i}" != "$-" ]; then . "$i" else . "$i" >/dev/null fi fi done unset i unset -f pathmunge export LANG=en_US.UTF-8 export HISTTIMEFORMAT="%F %T `whoami` " export JAVA_HOME=/usr/local/jdk export TOMCAT_HOME=/apps/tomcat export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$TOMCAT_HOME/bin:$PATH export CLASSPATH=.$CLASSPATH:$JAVA_HOME/lib:$JAVA_HOME/jre/lib:$JAVA_HOME/lib/tools.jar 1、编写dockerfile文件 root@master1:/opt/k8s-data/dockerfile/web/pub-images/jdk-1.8.212# vim Dockerfile FROM harbor.magedu.net/baseimages/magedu-centos-base:7.9.2009 MAINTAINER zhaoyang "2569220198@qq.com" ADD jdk-8u212-linux-x64.tar.gz /usr/local/src/ RUN ln -sv /usr/local/src/jdk1.8.0_212 /usr/local/jdk ADD profile /etc/profile ENV JAVA_HOME /usr/local/jdk ENV JRE_HOME $JAVA_HOME/jre ENV CLASSPATH $JAVA_HOME/lib/:$JRE_HOME/lib/ ENV PATH $PATH:$JAVA_HOME/bin 2、制作镜像，显示报错，报错截图如下 nerdctl build -t harbor.magedu.net/pub-images/jdk-base:v8.212 .
之前配置的证书是给containerd使用的，containerd可以上传下载镜像因此上传下载镜像是ok的；但是使用nerdctl命令build镜像的时候这个证书对nerdctl是无效的
查看containerd运行时的证书，是给containerd使用的，并非给nerdctl使用
nerdctl需要连接到harbor上下镜像的元数据；而如果harbor服务端使用的是https就会报错，而之前配置的harbor就是https的，那么现在可以把证书不放在harbor本地了，而是放到负载均衡器上，用nginx作为代理harbor的负载均衡器；把证书放到nginx上，harbor只需要支持http 80就行了；而nginx同时启动http 80,（用来下载镜像元数据），和https 443（用来推送镜像）
四、 基于nginx代理harbor并实现https
1、修改harbor配置文件
将harbor修改为http协议：
1.1、停止harbor服务
1.2、修改配置，重启服务
2、修改harbor.yaml文件 把https的配置全部注释掉 root@harbor:/apps/harbor# vim harbor.yml #https: # https port for harbor, default is 443 # port: 443 # The path of cert and key files for nginx # certificate: /apps/harbor/certs/magedu.net.crt # private_key: /apps/harbor/certs/magedu.net.key 3、重新加载harbor配置 root@harbor:/apps/harbor# ./prepare 4、重启服务 root@harbor:/apps/harbor# docker-compose up -d1.3、测试
此时在使用https就访问不了
在windows系统中hosts文件中添加域名ip, 使用域名访问可以进入harbor页面
可以看见是使用http访问
1、
2、
3、
但是k8s之前部署的环境都是走的https协议的harbor，因此现在k8s环境中的node节点是无法下载镜像的
1.4、nerdctl下载镜像报错
下图是node节点使用nerdctl下载镜像报错
五、解决报错
针对上面的问题有两种解决方案：
1、把containerd配置成使用http的下载方式
2、在harbor服务器前端加一个负载均衡器
1、在harbor前端加一个负载均衡
下面配置nginx负载均衡器，把etcd1主机作为nginx负载均衡
1、下载nginx root@etcd1:/usr/local/src# wget https://nginx.org/download/nginx-1.22.0.tar.gz 2、解压 root@etcd1:/usr/local/src# tar xvf nginx-1.22.0.tar.gz 3、安装pcre，gcc，zlib，g++,openssl依赖 3.1、pcre apt-get install libpcre3 libpcre3-dev 3.2、gcc g++ apt-get install build-essential apt-get install libtool apt install gcc 3.3、zlib apt-get install zlib1g-dev 3.4、openssl apt-get install openssl apt-get install libssl-dev 4、编译安装 root@etcd1:/usr/local/src# cd nginx-1.22.0 root@etcd1:/usr/local/src/nginx-1.22.0# ./configure --prefix=/apps/nginx \ > --with-http_ssl_module \ > --with-http_v2_module \ > --with-http_realip_module \ > --with-http_stub_status_module \ > --with-http_gzip_static_module \ > --with-pcre \ > --with-stream \ > --with-stream_ssl_module \ > --with-stream_realip_module 5、编译安装 root@etcd1:/usr/local/src/nginx-1.22.0# make && make install安装好nginx，把之前放在harbor上的证书放到nginx上
1、创建一个存放证书的目录 root@etcd1:/usr/local/src/nginx-1.22.0# mkdir /apps/nginx/certs -p 2、把harbor服务器上的证书拷贝到nginx上 这里harbor的证书，是在部署harbor的时候签发的，详情可见博客 https://blog.csdn.net/weixin_46476452/article/details/127732870 root@harbor:/apps/harbor/certs# scp magedu.net.crt magedu.net.key etcd1:/apps/nginx/certs 3、在nginx主机上查看下拷贝过来的证书文件 root@etcd1:/usr/local/src/nginx-1.22.0# ls /apps/nginx/certs/ magedu.net.crt magedu.net.key 4、配置nginx主配置文件，添加如下内容,把证书放到nginx root@etcd1:~# vim /apps/nginx/conf/nginx.conf 34 client_max_body_size 2000m; #客户端可以传输的最大文件大小，默认的大小只要几兆，上传下载镜像肯定是不行的，改成2G大小 35 server { 36 listen 80; #监听80 37 listen 443 ssl; #监听443 38 server_name harbor.magedu.net; #把harbor服务器的域名加载进来 39 ssl_certificate /apps/nginx/certs/magedu.net.crt; #证书路径 40 ssl_certificate_key /apps/nginx/certs/magedu.net.key; 41 ssl_session_cache shared:sslcache:20m; 42 ssl_session_timeout 10m; 43 44 #charset koi8-r; 45 46 #access_log logs/host.access.log main; 47 48 location / { 49 #root html; #location这里什么都不做，把这两行注释了 50 # index index.html index.htm; 51 proxy_pass http://172.31.7.104; #转到harbor服务器的ip上 52 } 53配置文件截图如下
测试下配置是否有误
启动nginx
root@etcd1:~# /apps/nginx/sbin/nginx root@etcd1:~# ss -tnl | grep 80 LISTEN 0 32768 172.31.7.106:2380 0.0.0.0:* LISTEN 0 511 0.0.0.0:80 0.0.0.0:*把k8s集群环境中各节点的/etc/hosts文件harbor服务器的域名地址换成 nginx服务器地址172.31.7.106
但是节点想要去harbor端下载镜像还需要harbor的证书，因此重新签发证书给各节点，除此之外还有个临时性解决方案，加一个参数--insecure-registry来信任我们自建的仓库
可以下载本地harbor上的镜像
构建镜像的主机进行一些配置
2、 配置buildkitd文件
root@master1:~# mkdir /etc/buildkit/
root@master1:~# vim /etc/buildkit/buildkitd.toml
[registry."harbor.magedu.net"] #对这个仓库使用http,和insecure
http = true
insecure = true
证书是我们自己签发的不被信任，使用https不被信任，所以使用http3、配置nerdctl文件
root@master1:~# cat /etc/nerdctl/nerdctl.toml
namespace = "k8s.io"
# k8s默认使用k8s.io namespace, nerdctl默认使用default namspace,因此需要切换至k8s.io的namespace才能显示对方的镜像，tag等
debug = false
debug_full = ture
insecure_registry = true
重启服务
root@master1:~# systemctl restart buildkitd.service
六、构建镜像
之前构建jdk镜像失败，现在可以正常构建了，以后上传下载镜像就通过负载均衡nginx了
查看下dockerfile
root@master1:/opt/k8s-data/dockerfile/web/pub-images/jdk-1.8.212# ls Dockerfile jdk-8u212-linux-x64.tar.gz profile root@master1:/opt/k8s-data/dockerfile/web/pub-images/jdk-1.8.212# cat Dockerfile FROM harbor.magedu.net/baseimages/magedu-centos-base:7.9.2009 MAINTAINER zhaoyang "2569220198@qq.com" ADD jdk-8u212-linux-x64.tar.gz /usr/local/src/ RUN ln -sv /usr/local/src/jdk1.8.0_212 /usr/local/jdk ADD profile /etc/profile ENV JAVA_HOME /usr/local/jdk ENV JRE_HOME $JAVA_HOME/jre ENV CLASSPATH $JAVA_HOME/lib/:$JRE_HOME/lib/ ENV PATH $PATH:$JAVA_HOME/bin 查看下profile文件内容 root@master1:/opt/k8s-data/dockerfile/web/pub-images/jdk-1.8.212# cat profile pathmunge () { case ":${PATH}:" in *:"$1":*) ;; *) if [ "$2" = "after" ] ; then PATH=$PATH:$1 else PATH=$1:$PATH fi esac } if [ -x /usr/bin/id ]; then if [ -z "$EUID" ]; then # ksh workaround EUID=`/usr/bin/id -u` UID=`/usr/bin/id -ru` fi USER="`/usr/bin/id -un`" LOGNAME=$USER MAIL="/var/spool/mail/$USER" fi # Path manipulation if [ "$EUID" = "0" ]; then pathmunge /usr/sbin pathmunge /usr/local/sbin else pathmunge /usr/local/sbin after pathmunge /usr/sbin after fi HOSTNAME=`/usr/bin/hostname 2>/dev/null` HISTSIZE=1000 if [ "$HISTCONTROL" = "ignorespace" ] ; then export HISTCONTROL=ignoreboth else export HISTCONTROL=ignoredups fi export PATH USER LOGNAME MAIL HOSTNAME HISTSIZE HISTCONTROL # By default, we want umask to get set. This sets it for login shell # Current threshold for system reserved uid/gids is 200 # You could check uidgid reservation validity in # /usr/share/doc/setup-*/uidgid file if [ $UID -gt 199 ] && [ "`/usr/bin/id -gn`" = "`/usr/bin/id -un`" ]; then umask 002 else umask 022 fi for i in /etc/profile.d/*.sh /etc/profile.d/sh.local ; do if [ -r "$i" ]; then if [ "${-#*i}" != "$-" ]; then . "$i" else . "$i" >/dev/null fi fi done unset i unset -f pathmunge export LANG=en_US.UTF-8 export HISTTIMEFORMAT="%F %T `whoami` " export JAVA_HOME=/usr/local/jdk export TOMCAT_HOME=/apps/tomcat export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$TOMCAT_HOME/bin:$PATH export CLASSPATH=.$CLASSPATH:$JAVA_HOME/lib:$JAVA_HOME/jre/lib:$JAVA_HOME/lib/tools.jar1、构建镜像
# nerdctl build -t harbor.magedu.net/pub-images/jdk-base:v8.212 .
2、 上传镜像到harbor
# nerdctl push harbor.magedu.net/pub-images/jdk-base:v8.212
因为前面的配置文件中/etc/nerdctl/nerdctl.toml开启了debug, 所以会输出很多信息，可以看见下面两个存放证书路径，现在可以把debug配置关闭了不需要回显过多的内容了，不然image查看镜像都会有很多debug信息
因此把证书拷贝到这个/etc/containerd/certs.d默认路径下就可以访问harbor了，就不需要使用加一个参数--insecure-registry的方式来访问了
3、再次构建镜像
基于刚刚打的jdk父镜像，二次制作tomcat镜像
root@master1:/opt/k8s-data/dockerfile/web/pub-images/tomcat-base-8.5.43# ls apache-tomcat-8.5.43.tar.gz build-command.sh Dockerfile 1、查看dockerfile root@master1:/opt/k8s-data/dockerfile/web/pub-images/tomcat-base-8.5.43# cat Dockerfile #Tomcat 8.5.43基础镜像 FROM harbor.magedu.net/pub-images/jdk-base:v8.212 MAINTAINER zhaoyang "2569220198@qq.com" RUN mkdir /apps /data/tomcat/webapps /data/tomcat/logs -pv ADD apache-tomcat-8.5.43.tar.gz /apps RUN useradd tomcat -u 2050 && ln -sv /apps/apache-tomcat-8.5.43 /apps/tomcat && chown -R tomcat.tomcat /apps /data -R root@master1:/opt/k8s-data/dockerfile/web/pub-images/tomcat-base-8.5.43# 2、查看build脚本 root@master1:/opt/k8s-data/dockerfile/web/pub-images/tomcat-base-8.5.43# cat build-command.sh #!/bin/bash nerdctl build -t harbor.magedu.net/pub-images/tomcat-base:v8.5.43 . nerdctl push harbor.magedu.net/pub-images/tomcat-base:v8.5.43 3、执行build脚本，打镜像并上传至harbor服务器上传的过程中显示443连不上，这个没关系，可以直接访问80端口，通过负载均衡器上传下载镜像
镜像已经构建成功，并且上传到了harbor了
下面进入tomcat容器检查下java环境是否正常
上面的镜像制作过程通过Dockerfile看出，基于magedu-centos-base:7.9.2009基础镜像构建出jdk-base:v8.212镜像，最后又基于dk-base:v8.212构建出tomcat-base:v8.5.43镜像
七、基于基础镜像，制作业务镜像
1、制作业务镜像
查看dockerfile同级所需文件
root@master1:/opt/k8s-data/dockerfile/web/magedu/tomcat-app1# ls
app1.tar.gz catalina.sh filebeat-7.5.1-x86_64.rpm index.html run_tomcat.sh
build-command.sh Dockerfile myapp server.xml
把所有脚本加上执行权限
root@master1:/opt/k8s-data/dockerfile/web/magedu/tomcat-app1# chmod a+x *.sh
这里的dockerfile文件是基于刚才的tomcat-base:v8.5.43镜像
root@master1:/opt/k8s-data/dockerfile/web/magedu/tomcat-app1# cat Dockerfile
FROM harbor.magedu.net/pub-images/tomcat-base:v8.5.43
ADD catalina.sh /apps/tomcat/bin/catalina.sh
ADD server.xml /apps/tomcat/conf/server.xml #通过server.xml指定tomcat webapps的目录
ADD app1.tar.gz /data/tomcat/webapps/myapp/ #把代码文件压缩包打进容器目录，这个目录是在server.xml中指定的
ADD run_tomcat.sh /apps/tomcat/bin/run_tomcat.sh
RUN chown -R nginx.nginx /data/ /apps/
EXPOSE 8080 8443
CMD ["/apps/tomcat/bin/run_tomcat.sh"]
2、查看下跟这个dockerfile文件中相关的其它文件
代码文件，是以压缩包的形式存在的然后直接在dockerfile中用add把压缩包直接打进容器的访问目录下自动解压，而如果代码文件是放在目录下，那么客户端访问这个代码时也需要加上这个目录
看下代码目录下的存放代码的index.html文件
root@master1:/opt/k8s-data/dockerfile/web/magedu/tomcat-app1# ls myapp/
app1.tar.gz index.html
删除文件
root@master1:/opt/k8s-data/dockerfile/web/magedu/tomcat-app1# rm -rf myapp/*
重新把代码文件的压缩包解压
root@master1:/opt/k8s-data/dockerfile/web/magedu/tomcat-app1# tar xvf app1.tar.gz -C myapp/
root@master1:/opt/k8s-data/dockerfile/web/magedu/tomcat-app1# ls myapp/
index.html
root@master1:/opt/k8s-data/dockerfile/web/magedu/tomcat-app1# cat index.html
tomcat app1 for linux n70
现在演示重新制作这个代码文件的压缩包
1、先看下现在是已经有了这个代码文件的压缩包app1.tar.gz了
root@master1:/opt/k8s-data/dockerfile/web/magedu/tomcat-app1# ls
app1.tar.gz catalina.sh filebeat-7.5.1-x86_64.rpm index.html run_tomcat.sh
build-command.sh Dockerfile filebeat.yml myapp server.xml
2、把app1.tar.gz包删除
root@master1:/opt/k8s-data/dockerfile/web/magedu/tomcat-app1# rm -rf app1.tar.gz
3、重新制作代码文件压缩包app1.tar.gz
进入存放代码文件的目录/myapp
root@master1:/opt/k8s-data/dockerfile/web/magedu/tomcat-app1# cd myapp/
root@master1:/opt/k8s-data/dockerfile/web/magedu/tomcat-app1/myapp# ls
index.html
查看代码文件内容
root@master1:/opt/k8s-data/dockerfile/web/magedu/tomcat-app1/myapp# cat index.html
tomcat app1 for linux n70
把代码文件进行打包，再次命名为app1.tar.gz，（因为dockerfile文件中已经定义了这个名字）
root@master1:/opt/k8s-data/dockerfile/web/magedu/tomcat-app1/myapp# tar czf app1.tar.gz
把代码文件压缩包拷贝到与dockerfile文件同一级目录下
root@master1:/opt/k8s-data/dockerfile/web/magedu/tomcat-app1/myapp# cd ..
root@master1:/opt/k8s-data/dockerfile/web/magedu/tomcat-app1# ls | grep app1.tar.gz
root@master1:/opt/k8s-data/dockerfile/web/magedu/tomcat-app1# mv myapp/app1.tar.gz .
root@master1:/opt/k8s-data/dockerfile/web/magedu/tomcat-app1# ls | grep app1.tar.gz
app1.tar.gz
通过server.xml文件来指定，server.xml是tomcat主配置文件
看下service.xml内容
root@master1:/opt/k8s-data/dockerfile/web/magedu/tomcat-app1# cat server.xml
<?xml version='1.0' encoding='utf-8'?>
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<!-- Note: A "Server" is not itself a "Container", so you may not
define subcomponents such as "Valves" at this level.
Documentation at /docs/config/server.html
-->
<Server port="8005" shutdown="SHUTDOWN">
<Listener className="org.apache.catalina.startup.VersionLoggerListener" />
<!-- Security listener. Documentation at /docs/config/listeners.html
<Listener className="org.apache.catalina.security.SecurityListener" />
-->
<!--APR library loader. Documentation at /docs/apr.html -->
<Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />
<!-- Prevent memory leaks due to use of particular java/javax APIs-->
<Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
<Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
<Listener className="org.apache.catalina.core.ThreadLocalLeakPreventionListener" />
<!-- Global JNDI resources
Documentation at /docs/jndi-resources-howto.html
-->
<GlobalNamingResources>
<!-- Editable user database that can also be used by
UserDatabaseRealm to authenticate users
-->
<Resource name="UserDatabase" auth="Container"
type="org.apache.catalina.UserDatabase"
description="User database that can be updated and saved"
factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
pathname="conf/tomcat-users.xml" />
</GlobalNamingResources>
<!-- A "Service" is a collection of one or more "Connectors" that share
a single "Container" Note: A "Service" is not itself a "Container",
so you may not define subcomponents such as "Valves" at this level.
Documentation at /docs/config/service.html
-->
<Service name="Catalina">
<!--The connectors can use a shared executor, you can define one or more named thread pools-->
<!--
<Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
maxThreads="150" minSpareThreads="4"/>
-->
<!-- A "Connector" represents an endpoint by which requests are received
and responses are returned. Documentation at :
Java HTTP Connector: /docs/config/http.html (blocking & non-blocking)
Java AJP Connector: /docs/config/ajp.html
APR (HTTP/AJP) Connector: /docs/apr.html
Define a non-SSL/TLS HTTP/1.1 Connector on port 8080
-->
<Connector port="8080" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8443" />
<!-- A "Connector" using the shared thread pool-->
<!--
<Connector executor="tomcatThreadPool"
port="8080" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8443" />
-->
<!-- Define a SSL/TLS HTTP/1.1 Connector on port 8443
This connector uses the NIO implementation that requires the JSSE
style configuration. When using the APR/native implementation, the
OpenSSL style configuration is required as described in the APR/native
documentation -->
<!--
<Connector port="8443" protocol="org.apache.coyote.http11.Http11NioProtocol"
maxThreads="150" SSLEnabled="true" scheme="https" secure="true"
clientAuth="false" sslProtocol="TLS" />
-->
<!-- Define an AJP 1.3 Connector on port 8009 -->
<Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />
<!-- An Engine represents the entry point (within Catalina) that processes
every request. The Engine implementation for Tomcat stand alone
analyzes the HTTP headers included with the request, and passes them
on to the appropriate Host (virtual host).
Documentation at /docs/config/engine.html -->
<!-- You should set jvmRoute to support load-balancing via AJP ie :
<Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm1">
-->
<Engine name="Catalina" defaultHost="localhost">
<!--For clustering, please take a look at documentation at:
/docs/cluster-howto.html (simple how to)
/docs/config/cluster.html (reference documentation) -->
<!--
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"/>
-->
<!-- Use the LockOutRealm to prevent attempts to guess user passwords
via a brute-force attack -->
<Realm className="org.apache.catalina.realm.LockOutRealm">
<!-- This Realm uses the UserDatabase configured in the global JNDI
resources under the key "UserDatabase". Any edits
that are performed against this UserDatabase are immediately
available for use by the Realm. -->
<Realm className="org.apache.catalina.realm.UserDatabaseRealm"
resourceName="UserDatabase"/>
</Realm>
<Host name="localhost" appBase="/data/tomcat/webapps" unpackWARs="false" autoDeploy="false">
#appBase路径指向了/data/tomcat/webapps，这个目录在上一次打tomcat镜像时候已经打好了
<!-- SingleSignOn valve, share authentication between web applications
Documentation at: /docs/config/valve.html -->
<!--
<Valve className="org.apache.catalina.authenticator.SingleSignOn" />
-->
<!-- Access log processes all example.
Documentation at: /docs/config/valve.html
Note: The pattern used is equivalent to using pattern="common" -->
<Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
prefix="localhost_access_log" suffix=".txt"
pattern="%h %l %u %t "%r" %s %b" />
</Host>
</Engine>
</Service>
</Server>
catalina.sh是tomcat的启动脚本
看下catalina.sh内容
root@master1:/opt/k8s-data/dockerfile/web/magedu/tomcat-app1# cat catalina.sh
#!/bin/sh
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# -----------------------------------------------------------------------------
# Control Script for the CATALINA Server
#
# Environment Variable Prerequisites
#
# Do not set the variables in this script. Instead put them into a script
# setenv.sh in CATALINA_BASE/bin to keep your customizations separate.
#
# CATALINA_HOME May point at your Catalina "build" directory.
#
# CATALINA_BASE (Optional) Base directory for resolving dynamic portions
# of a Catalina installation. If not present, resolves to
# the same directory that CATALINA_HOME points to.
#
# CATALINA_OUT (Optional) Full path to a file where stdout and stderr
# will be redirected.
# Default is $CATALINA_BASE/logs/catalina.out
#
# CATALINA_OPTS (Optional) Java runtime options used when the "start",
# "run" or "debug" command is executed.
# Include here and not in JAVA_OPTS all options, that should
# only be used by Tomcat itself, not by the stop process,
# the version command etc.
# Examples are heap size, GC logging, JMX ports etc.
#
# CATALINA_TMPDIR (Optional) Directory path location of temporary directory
# the JVM should use (java.io.tmpdir). Defaults to
# $CATALINA_BASE/temp.
#
# JAVA_HOME Must point at your Java Development Kit installation.
# Required to run the with the "debug" argument.
#
# JRE_HOME Must point at your Java Runtime installation.
# Defaults to JAVA_HOME if empty. If JRE_HOME and JAVA_HOME
# are both set, JRE_HOME is used.
#
# JAVA_OPTS (Optional) Java runtime options used when any command
# is executed.
# Include here and not in CATALINA_OPTS all options, that
# should be used by Tomcat and also by the stop process,
# the version command etc.
# Most options should go into CATALINA_OPTS.
#
# JAVA_ENDORSED_DIRS (Optional) Lists of of colon separated directories
# containing some jars in order to allow replacement of APIs
# created outside of the JCP (i.e. DOM and SAX from W3C).
# It can also be used to update the XML parser implementation.
# Note that Java 9 no longer supports this feature.
# Defaults to $CATALINA_HOME/endorsed.
#
# JPDA_TRANSPORT (Optional) JPDA transport used when the "jpda start"
# command is executed. The default is "dt_socket".
#
# JPDA_ADDRESS (Optional) Java runtime options used when the "jpda start"
# command is executed. The default is localhost:8000.
#
# JPDA_SUSPEND (Optional) Java runtime options used when the "jpda start"
# command is executed. Specifies whether JVM should suspend
# execution immediately after startup. Default is "n".
#
# JPDA_OPTS (Optional) Java runtime options used when the "jpda start"
# command is executed. If used, JPDA_TRANSPORT, JPDA_ADDRESS,
# and JPDA_SUSPEND are ignored. Thus, all required jpda
# options MUST be specified. The default is:
#
# -agentlib:jdwp=transport=$JPDA_TRANSPORT,
# address=$JPDA_ADDRESS,server=y,suspend=$JPDA_SUSPEND
#
# JSSE_OPTS (Optional) Java runtime options used to control the TLS
# implementation when JSSE is used. Default is:
# "-Djdk.tls.ephemeralDHKeySize=2048"
#
# CATALINA_PID (Optional) Path of the file which should contains the pid
# of the catalina startup java process, when start (fork) is
# used
#
# LOGGING_CONFIG (Optional) Override Tomcat's logging config file
# Example (all one line)
# LOGGING_CONFIG="-Djava.util.logging.config.file=$CATALINA_BASE/conf/logging.properties"
#
# LOGGING_MANAGER (Optional) Override Tomcat's logging manager
# Example (all one line)
# LOGGING_MANAGER="-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager"
#
# USE_NOHUP (Optional) If set to the string true the start command will
# use nohup so that the Tomcat process will ignore any hangup
# signals. Default is "false" unless running on HP-UX in which
# case the default is "true"
# -----------------------------------------------------------------------------
JAVA_OPTS="-server -Xms1g -Xmx1g -Xss512k -Xmn1g -XX:CMSInitiatingOccupancyFraction=65 -XX:+UseFastAccessorMethods -XX:+AggressiveOpts -XX:+UseBiasedLocking -XX:+DisableExplicitGC -XX:MaxTenuringThreshold=10 -XX:NewSize=2048M -XX:MaxNewSize=2048M -XX:NewRatio=2 -XX:PermSize=128m -XX:MaxPermSize=512m -XX:CMSFullGCsBeforeCompaction=5 -XX:+ExplicitGCInvokesConcurrent -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSParallelRemarkEnabled"
# OS specific support. $var _must_ be set to either true or false.
cygwin=false
darwin=false
os400=false
hpux=false
case "`uname`" in
CYGWIN*) cygwin=true;;
Darwin*) darwin=true;;
OS400*) os400=true;;
HP-UX*) hpux=true;;
esac
# resolve links - $0 may be a softlink
PRG="$0"
while [ -h "$PRG" ]; do
ls=`ls -ld "$PRG"`
link=`expr "$ls" : '.*-> \(.*\)$'`
if expr "$link" : '/.*' > /dev/null; then
PRG="$link"
else
PRG=`dirname "$PRG"`/"$link"
fi
done
# Get standard environment variables
PRGDIR=`dirname "$PRG"`
# Only set CATALINA_HOME if not already set
[ -z "$CATALINA_HOME" ] && CATALINA_HOME=`cd "$PRGDIR/.." >/dev/null; pwd`
# Copy CATALINA_BASE from CATALINA_HOME if not already set
[ -z "$CATALINA_BASE" ] && CATALINA_BASE="$CATALINA_HOME"
# Ensure that any user defined CLASSPATH variables are not used on startup,
# but allow them to be specified in setenv.sh, in rare case when it is needed.
CLASSPATH=
if [ -r "$CATALINA_BASE/bin/setenv.sh" ]; then
. "$CATALINA_BASE/bin/setenv.sh"
elif [ -r "$CATALINA_HOME/bin/setenv.sh" ]; then
. "$CATALINA_HOME/bin/setenv.sh"
fi
# For Cygwin, ensure paths are in UNIX format before anything is touched
if $cygwin; then
[ -n "$JAVA_HOME" ] && JAVA_HOME=`cygpath --unix "$JAVA_HOME"`
[ -n "$JRE_HOME" ] && JRE_HOME=`cygpath --unix "$JRE_HOME"`
[ -n "$CATALINA_HOME" ] && CATALINA_HOME=`cygpath --unix "$CATALINA_HOME"`
[ -n "$CATALINA_BASE" ] && CATALINA_BASE=`cygpath --unix "$CATALINA_BASE"`
[ -n "$CLASSPATH" ] && CLASSPATH=`cygpath --path --unix "$CLASSPATH"`
fi
# Ensure that neither CATALINA_HOME nor CATALINA_BASE contains a colon
# as this is used as the separator in the classpath and Java provides no
# mechanism for escaping if the same character appears in the path.
case $CATALINA_HOME in
*:*) echo "Using CATALINA_HOME: $CATALINA_HOME";
echo "Unable to start as CATALINA_HOME contains a colon (:) character";
exit 1;
esac
case $CATALINA_BASE in
*:*) echo "Using CATALINA_BASE: $CATALINA_BASE";
echo "Unable to start as CATALINA_BASE contains a colon (:) character";
exit 1;
esac
# For OS400
if $os400; then
# Set job priority to standard for interactive (interactive - 6) by using
# the interactive priority - 6, the helper threads that respond to requests
# will be running at the same priority as interactive jobs.
COMMAND='chgjob job('$JOBNAME') runpty(6)'
system $COMMAND
# Enable multi threading
export QIBM_MULTI_THREADED=Y
fi
# Get standard Java environment variables
if $os400; then
# -r will Only work on the os400 if the files are:
# 1. owned by the user
# 2. owned by the PRIMARY group of the user
# this will not work if the user belongs in secondary groups
. "$CATALINA_HOME"/bin/setclasspath.sh
else
if [ -r "$CATALINA_HOME"/bin/setclasspath.sh ]; then
. "$CATALINA_HOME"/bin/setclasspath.sh
else
echo "Cannot find $CATALINA_HOME/bin/setclasspath.sh"
echo "This file is needed to run this program"
exit 1
fi
fi
# Add on extra jar files to CLASSPATH
if [ ! -z "$CLASSPATH" ] ; then
CLASSPATH="$CLASSPATH":
fi
CLASSPATH="$CLASSPATH""$CATALINA_HOME"/bin/bootstrap.jar
if [ -z "$CATALINA_OUT" ] ; then
CATALINA_OUT="$CATALINA_BASE"/logs/catalina.out
fi
if [ -z "$CATALINA_TMPDIR" ] ; then
# Define the java.io.tmpdir to use for Catalina
CATALINA_TMPDIR="$CATALINA_BASE"/temp
fi
# Add tomcat-juli.jar to classpath
# tomcat-juli.jar can be over-ridden per instance
if [ -r "$CATALINA_BASE/bin/tomcat-juli.jar" ] ; then
CLASSPATH=$CLASSPATH:$CATALINA_BASE/bin/tomcat-juli.jar
else
CLASSPATH=$CLASSPATH:$CATALINA_HOME/bin/tomcat-juli.jar
fi
# Bugzilla 37848: When no TTY is available, don't output to console
have_tty=0
if [ "`tty`" != "not a tty" ]; then
have_tty=1
fi
# For Cygwin, switch paths to Windows format before running java
if $cygwin; then
JAVA_HOME=`cygpath --absolute --windows "$JAVA_HOME"`
JRE_HOME=`cygpath --absolute --windows "$JRE_HOME"`
CATALINA_HOME=`cygpath --absolute --windows "$CATALINA_HOME"`
CATALINA_BASE=`cygpath --absolute --windows "$CATALINA_BASE"`
CATALINA_TMPDIR=`cygpath --absolute --windows "$CATALINA_TMPDIR"`
CLASSPATH=`cygpath --path --windows "$CLASSPATH"`
JAVA_ENDORSED_DIRS=`cygpath --path --windows "$JAVA_ENDORSED_DIRS"`
fi
if [ -z "$JSSE_OPTS" ] ; then
JSSE_OPTS="-Djdk.tls.ephemeralDHKeySize=2048"
fi
JAVA_OPTS="$JAVA_OPTS $JSSE_OPTS"
# Register custom URL handlers
# Do this here so custom URL handles (specifically 'war:...') can be used in the security policy
JAVA_OPTS="$JAVA_OPTS -Djava.protocol.handler.pkgs=org.apache.catalina.webresources"
# Set juli LogManager config file if it is present and an override has not been issued
if [ -z "$LOGGING_CONFIG" ]; then
if [ -r "$CATALINA_BASE"/conf/logging.properties ]; then
LOGGING_CONFIG="-Djava.util.logging.config.file=$CATALINA_BASE/conf/logging.properties"
else
# Bugzilla 45585
LOGGING_CONFIG="-Dnop"
fi
fi
if [ -z "$LOGGING_MANAGER" ]; then
LOGGING_MANAGER="-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager"
fi
# Java 9 no longer supports the java.endorsed.dirs
# system property. Only try to use it if
# JAVA_ENDORSED_DIRS was explicitly set
# or CATALINA_HOME/endorsed exists.
ENDORSED_PROP=ignore.endorsed.dirs
if [ -n "$JAVA_ENDORSED_DIRS" ]; then
ENDORSED_PROP=java.endorsed.dirs
fi
if [ -d "$CATALINA_HOME/endorsed" ]; then
ENDORSED_PROP=java.endorsed.dirs
fi
# Uncomment the following line to make the umask available when using the
# org.apache.catalina.security.SecurityListener
#JAVA_OPTS="$JAVA_OPTS -Dorg.apache.catalina.security.SecurityListener.UMASK=`umask`"
if [ -z "$USE_NOHUP" ]; then
if $hpux; then
USE_NOHUP="true"
else
USE_NOHUP="false"
fi
fi
unset _NOHUP
if [ "$USE_NOHUP" = "true" ]; then
_NOHUP=nohup
fi
# Add the JAVA 9 specific start-up parameters required by Tomcat
JDK_JAVA_OPTIONS="$JDK_JAVA_OPTIONS --add-opens=java.base/java.lang=ALL-UNNAMED"
JDK_JAVA_OPTIONS="$JDK_JAVA_OPTIONS --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED"
export JDK_JAVA_OPTIONS
# ----- Execute The Requested Command -----------------------------------------
# Bugzilla 37848: only output this if we have a TTY
if [ $have_tty -eq 1 ]; then
echo "Using CATALINA_BASE: $CATALINA_BASE"
echo "Using CATALINA_HOME: $CATALINA_HOME"
echo "Using CATALINA_TMPDIR: $CATALINA_TMPDIR"
if [ "$1" = "debug" ] ; then
echo "Using JAVA_HOME: $JAVA_HOME"
else
echo "Using JRE_HOME: $JRE_HOME"
fi
echo "Using CLASSPATH: $CLASSPATH"
if [ ! -z "$CATALINA_PID" ]; then
echo "Using CATALINA_PID: $CATALINA_PID"
fi
fi
if [ "$1" = "jpda" ] ; then
if [ -z "$JPDA_TRANSPORT" ]; then
JPDA_TRANSPORT="dt_socket"
fi
if [ -z "$JPDA_ADDRESS" ]; then
JPDA_ADDRESS="localhost:8000"
fi
if [ -z "$JPDA_SUSPEND" ]; then
JPDA_SUSPEND="n"
fi
if [ -z "$JPDA_OPTS" ]; then
JPDA_OPTS="-agentlib:jdwp=transport=$JPDA_TRANSPORT,address=$JPDA_ADDRESS,server=y,suspend=$JPDA_SUSPEND"
fi
CATALINA_OPTS="$JPDA_OPTS $CATALINA_OPTS"
shift
fi
if [ "$1" = "debug" ] ; then
if $os400; then
echo "Debug command not available on OS400"
exit 1
else
shift
if [ "$1" = "-security" ] ; then
if [ $have_tty -eq 1 ]; then
echo "Using Security Manager"
fi
shift
exec "$_RUNJDB" "$LOGGING_CONFIG" $LOGGING_MANAGER $JAVA_OPTS $CATALINA_OPTS \
-D$ENDORSED_PROP="$JAVA_ENDORSED_DIRS" \
-classpath "$CLASSPATH" \
-sourcepath "$CATALINA_HOME"/../../java \
-Djava.security.manager \
-Djava.security.policy=="$CATALINA_BASE"/conf/catalina.policy \
-Dcatalina.base="$CATALINA_BASE" \
-Dcatalina.home="$CATALINA_HOME" \
-Djava.io.tmpdir="$CATALINA_TMPDIR" \
org.apache.catalina.startup.Bootstrap "$@" start
else
exec "$_RUNJDB" "$LOGGING_CONFIG" $LOGGING_MANAGER $JAVA_OPTS $CATALINA_OPTS \
-D$ENDORSED_PROP="$JAVA_ENDORSED_DIRS" \
-classpath "$CLASSPATH" \
-sourcepath "$CATALINA_HOME"/../../java \
-Dcatalina.base="$CATALINA_BASE" \
-Dcatalina.home="$CATALINA_HOME" \
-Djava.io.tmpdir="$CATALINA_TMPDIR" \
org.apache.catalina.startup.Bootstrap "$@" start
fi
fi
elif [ "$1" = "run" ]; then
shift
if [ "$1" = "-security" ] ; then
if [ $have_tty -eq 1 ]; then
echo "Using Security Manager"
fi
shift
eval exec "\"$_RUNJAVA\"" "\"$LOGGING_CONFIG\"" $LOGGING_MANAGER $JAVA_OPTS $CATALINA_OPTS \
-D$ENDORSED_PROP="\"$JAVA_ENDORSED_DIRS\"" \
-classpath "\"$CLASSPATH\"" \
-Djava.security.manager \
-Djava.security.policy=="\"$CATALINA_BASE/conf/catalina.policy\"" \
-Dcatalina.base="\"$CATALINA_BASE\"" \
-Dcatalina.home="\"$CATALINA_HOME\"" \
-Djava.io.tmpdir="\"$CATALINA_TMPDIR\"" \
org.apache.catalina.startup.Bootstrap "$@" start
else
eval exec "\"$_RUNJAVA\"" "\"$LOGGING_CONFIG\"" $LOGGING_MANAGER $JAVA_OPTS $CATALINA_OPTS \
-D$ENDORSED_PROP="\"$JAVA_ENDORSED_DIRS\"" \
-classpath "\"$CLASSPATH\"" \
-Dcatalina.base="\"$CATALINA_BASE\"" \
-Dcatalina.home="\"$CATALINA_HOME\"" \
-Djava.io.tmpdir="\"$CATALINA_TMPDIR\"" \
org.apache.catalina.startup.Bootstrap "$@" start
fi
elif [ "$1" = "start" ] ; then
if [ ! -z "$CATALINA_PID" ]; then
if [ -f "$CATALINA_PID" ]; then
if [ -s "$CATALINA_PID" ]; then
echo "Existing PID file found during start."
if [ -r "$CATALINA_PID" ]; then
PID=`cat "$CATALINA_PID"`
ps -p $PID >/dev/null 2>&1
if [ $? -eq 0 ] ; then
echo "Tomcat appears to still be running with PID $PID. Start aborted."
echo "If the following process is not a Tomcat process, remove the PID file and try again:"
ps -f -p $PID
exit 1
else
echo "Removing/clearing stale PID file."
rm -f "$CATALINA_PID" >/dev/null 2>&1
if [ $? != 0 ]; then
if [ -w "$CATALINA_PID" ]; then
cat /dev/null > "$CATALINA_PID"
else
echo "Unable to remove or clear stale PID file. Start aborted."
exit 1
fi
fi
fi
else
echo "Unable to read PID file. Start aborted."
exit 1
fi
else
rm -f "$CATALINA_PID" >/dev/null 2>&1
if [ $? != 0 ]; then
if [ ! -w "$CATALINA_PID" ]; then
echo "Unable to remove or write to empty PID file. Start aborted."
exit 1
fi
fi
fi
fi
fi
shift
touch "$CATALINA_OUT"
if [ "$1" = "-security" ] ; then
if [ $have_tty -eq 1 ]; then
echo "Using Security Manager"
fi
shift
eval $_NOHUP "\"$_RUNJAVA\"" "\"$LOGGING_CONFIG\"" $LOGGING_MANAGER $JAVA_OPTS $CATALINA_OPTS \
-D$ENDORSED_PROP="\"$JAVA_ENDORSED_DIRS\"" \
-classpath "\"$CLASSPATH\"" \
-Djava.security.manager \
-Djava.security.policy=="\"$CATALINA_BASE/conf/catalina.policy\"" \
-Dcatalina.base="\"$CATALINA_BASE\"" \
-Dcatalina.home="\"$CATALINA_HOME\"" \
-Djava.io.tmpdir="\"$CATALINA_TMPDIR\"" \
org.apache.catalina.startup.Bootstrap "$@" start \
>> "$CATALINA_OUT" 2>&1 "&"
else
eval $_NOHUP "\"$_RUNJAVA\"" "\"$LOGGING_CONFIG\"" $LOGGING_MANAGER $JAVA_OPTS $CATALINA_OPTS \
-D$ENDORSED_PROP="\"$JAVA_ENDORSED_DIRS\"" \
-classpath "\"$CLASSPATH\"" \
-Dcatalina.base="\"$CATALINA_BASE\"" \
-Dcatalina.home="\"$CATALINA_HOME\"" \
-Djava.io.tmpdir="\"$CATALINA_TMPDIR\"" \
org.apache.catalina.startup.Bootstrap "$@" start \
>> "$CATALINA_OUT" 2>&1 "&"
fi
if [ ! -z "$CATALINA_PID" ]; then
echo $! > "$CATALINA_PID"
fi
echo "Tomcat started."
elif [ "$1" = "stop" ] ; then
shift
SLEEP=5
if [ ! -z "$1" ]; then
echo $1 | grep "[^0-9]" >/dev/null 2>&1
if [ $? -gt 0 ]; then
SLEEP=$1
shift
fi
fi
FORCE=0
if [ "$1" = "-force" ]; then
shift
FORCE=1
fi
if [ ! -z "$CATALINA_PID" ]; then
if [ -f "$CATALINA_PID" ]; then
if [ -s "$CATALINA_PID" ]; then
kill -0 `cat "$CATALINA_PID"` >/dev/null 2>&1
if [ $? -gt 0 ]; then
echo "PID file found but no matching process was found. Stop aborted."
exit 1
fi
else
echo "PID file is empty and has been ignored."
fi
else
echo "\$CATALINA_PID was set but the specified file does not exist. Is Tomcat running? Stop aborted."
exit 1
fi
fi
eval "\"$_RUNJAVA\"" $LOGGING_MANAGER $JAVA_OPTS \
-D$ENDORSED_PROP="\"$JAVA_ENDORSED_DIRS\"" \
-classpath "\"$CLASSPATH\"" \
-Dcatalina.base="\"$CATALINA_BASE\"" \
-Dcatalina.home="\"$CATALINA_HOME\"" \
-Djava.io.tmpdir="\"$CATALINA_TMPDIR\"" \
org.apache.catalina.startup.Bootstrap "$@" stop
# stop failed. Shutdown port disabled? Try a normal kill.
if [ $? != 0 ]; then
if [ ! -z "$CATALINA_PID" ]; then
echo "The stop command failed. Attempting to signal the process to stop through OS signal."
kill -15 `cat "$CATALINA_PID"` >/dev/null 2>&1
fi
fi
if [ ! -z "$CATALINA_PID" ]; then
if [ -f "$CATALINA_PID" ]; then
while [ $SLEEP -ge 0 ]; do
kill -0 `cat "$CATALINA_PID"` >/dev/null 2>&1
if [ $? -gt 0 ]; then
rm -f "$CATALINA_PID" >/dev/null 2>&1
if [ $? != 0 ]; then
if [ -w "$CATALINA_PID" ]; then
cat /dev/null > "$CATALINA_PID"
# If Tomcat has stopped don't try and force a stop with an empty PID file
FORCE=0
else
echo "The PID file could not be removed or cleared."
fi
fi
echo "Tomcat stopped."
break
fi
if [ $SLEEP -gt 0 ]; then
sleep 1
fi
if [ $SLEEP -eq 0 ]; then
echo "Tomcat did not stop in time."
if [ $FORCE -eq 0 ]; then
echo "PID file was not removed."
fi
echo "To aid diagnostics a thread dump has been written to standard out."
kill -3 `cat "$CATALINA_PID"`
fi
SLEEP=`expr $SLEEP - 1 `
done
fi
fi
KILL_SLEEP_INTERVAL=5
if [ $FORCE -eq 1 ]; then
if [ -z "$CATALINA_PID" ]; then
echo "Kill failed: \$CATALINA_PID not set"
else
if [ -f "$CATALINA_PID" ]; then
PID=`cat "$CATALINA_PID"`
echo "Killing Tomcat with the PID: $PID"
kill -9 $PID
while [ $KILL_SLEEP_INTERVAL -ge 0 ]; do
kill -0 `cat "$CATALINA_PID"` >/dev/null 2>&1
if [ $? -gt 0 ]; then
rm -f "$CATALINA_PID" >/dev/null 2>&1
if [ $? != 0 ]; then
if [ -w "$CATALINA_PID" ]; then
cat /dev/null > "$CATALINA_PID"
else
echo "The PID file could not be removed."
fi
fi
echo "The Tomcat process has been killed."
break
fi
if [ $KILL_SLEEP_INTERVAL -gt 0 ]; then
sleep 1
fi
KILL_SLEEP_INTERVAL=`expr $KILL_SLEEP_INTERVAL - 1 `
done
if [ $KILL_SLEEP_INTERVAL -lt 0 ]; then
echo "Tomcat has not been killed completely yet. The process might be waiting on some system call or might be UNINTERRUPTIBLE."
fi
fi
fi
fi
elif [ "$1" = "configtest" ] ; then
eval "\"$_RUNJAVA\"" $LOGGING_MANAGER $JAVA_OPTS \
-D$ENDORSED_PROP="\"$JAVA_ENDORSED_DIRS\"" \
-classpath "\"$CLASSPATH\"" \
-Dcatalina.base="\"$CATALINA_BASE\"" \
-Dcatalina.home="\"$CATALINA_HOME\"" \
-Djava.io.tmpdir="\"$CATALINA_TMPDIR\"" \
org.apache.catalina.startup.Bootstrap configtest
result=$?
if [ $result -ne 0 ]; then
echo "Configuration error detected!"
fi
exit $result
elif [ "$1" = "version" ] ; then
"$_RUNJAVA" \
-classpath "$CATALINA_HOME/lib/catalina.jar" \
org.apache.catalina.util.ServerInfo
else
echo "Usage: catalina.sh ( commands ... )"
echo "commands:"
if $os400; then
echo " debug Start Catalina in a debugger (not available on OS400)"
echo " debug -security Debug Catalina with a security manager (not available on OS400)"
else
echo " debug Start Catalina in a debugger"
echo " debug -security Debug Catalina with a security manager"
fi
echo " jpda start Start Catalina under JPDA debugger"
echo " run Start Catalina in the current window"
echo " run -security Start in the current window with security manager"
echo " start Start Catalina in a separate window"
echo " start -security Start in a separate window with security manager"
echo " stop Stop Catalina, waiting up to 5 seconds for the process to end"
echo " stop n Stop Catalina, waiting up to n seconds for the process to end"
echo " stop -force Stop Catalina, wait up to 5 seconds and then use kill -KILL if still running"
echo " stop n -force Stop Catalina, wait up to n seconds and then use kill -KILL if still running"
echo " configtest Run a basic syntax check on server.xml - check exit code for result"
echo " version What version of tomcat are you running?"
echo "Note: Waiting for the process to end and use of the -force option require that \$CATALINA_PID is defined"
exit 1
fi
3、构建镜像
把dockerfile相关文件介绍清楚后，现在开始构建镜像
这里使用脚本构建，查看脚本内容，这里镜像的tag可以自己在脚本后面传递参数定义
root@master1:/opt/k8s-data/dockerfile/web/magedu/tomcat-app1# cat build-command.sh #!/bin/bash TAG=$1 nerdctl build -t harbor.magedu.net/magedu/tomcat-app1:${TAG} . nerdctl push harbor.magedu.net/magedu/tomcat-app1:${TAG} root@master1:/opt/k8s-data/dockerfile/web/magedu/tomcat-app1# bash build-command.sh v1查看本地镜像已经构建完成了
查看镜像已经把这个业务镜像上传至harbor服务器上了
至此就把业务镜像tomcat准备好了，就可以把它运行在K8S上了
4、测试镜像可用性
1.1、下面先测试下这个镜像是否正常可用
1.2、访问一下master1节点的地址，显示可以访问
访问myapp
测试说明镜像是没问题的，下面就可以把它运行在K8S上
八、基于业务镜像，运行tomcat后端业务
1、部署tomcat容器
1、编写yaml文件 root@master1:~# vim tomcat-app1.yaml kind: Deployment apiVersion: apps/v1 metadata: labels: app: tomcat-deployment-label name: tomcat-deployment namespace: myserver spec: replicas: 1 selector: matchLabels: app: tomcat-selector template: metadata: labels: app: tomcat-selector spec: containers: - name: tomcat-app1-container image: harbor.magedu.net/magedu/tomcat-app1:v1 imagePullPolicy: IfNotPresent ports: - containerPort: 8080 protocol: TCP name: http env: - name: "password" value: "123456" - name: "age" value: "18" volumeMounts: - name: images mountPath: /usr/local/nginx/html/webapp/images readOnly: false - name: static mountPath: /usr/local/nginx/html/webapp/static readOnly: false volumes: - name: images nfs: server: 172.31.7.109 path: /data/k8sdata/magedu/images - name: static nfs: server: 172.31.7.109 path: /data/k8sdata/magedu/static --- kind: Service apiVersion: v1 metadata: labels: app: tomcat-service-label name: tomcat-service namespace: myserver spec: type: NodePort ports: - name: http port: 80 protocol: TCP targetPort: 8080 nodePort: 30092 selector: app: tomcat-selector 2、yaml文件中存储卷，在172.31.7.109主机作为共享存储服务器 配置共享存储服务器配置参照博客 https://blog.csdn.net/weixin_46476452/article/details/127985493 第三步、nfs共享存储内容 创建共享目录 root@haproxy02:~# mkdir -p /data/k8sdata/magedu/images -p root@haproxy02:~# mkdir -p /data/k8sdata/magedu/static配置生效，查看/etc/export
到客户master端查验
创建资源 root@master1:~# kubectl apply -f tomcat-app1.yaml deployment.apps/tomcat-deployment created service/tomcat-service created
2、查看资源
查看pod资源和service资源都已经创建成功
访问下node节点的30092端口就可以访问到tomcat pod的80端口服务了
再访问下tomcat的首页，这个首页文件就相当于开发写的代码包，运维只负责把代码包部署起来；到此为止就已经把后端服务部署起来了，后面再部署一个前端服务
九、制作nginx前端业务镜像，运行nginx前端服务
1、制作nginx基础镜像
1、制作基础镜像
root@master1:~# ls build-command.sh Dockerfile nginx-1.22.0.tar.gz 1、编写dockerfile文件 root@master1:~# cat Dockerfile #Nginx Base Image FROM harbor.magedu.net/baseimages/magedu-centos-base:7.9.2009 #这个是基础镜像，安装好了编译环境的 MAINTAINER 2569220198@qq.com RUN yum install -y vim wget tree lrzsz gcc gcc-c++ automake pcre pcre-devel zlib zlib-devel openssl openssl-devel iproute net-tools iotop ADD nginx-1.22.0.tar.gz /usr/local/src/ RUN cd /usr/local/src/nginx-1.22.0 && ./configure && make && make install && ln -sv /usr/local/nginx/sbin/nginx /usr/sbin/nginx &&rm -rf /usr/local/src/nginx-1.22.0.tar.gz 2、编写构建镜像上传镜像脚本 root@master1:~# cat build-command.sh #!/bin/bash nerdctl build -t harbor.magedu.net/pub-images/nginx-base:v1.22.0 . nerdctl push harbor.magedu.net/pub-images/nginx-base:v1.22.0 3、下载nginx-1.22.0.tar.gz二进制包到当前目录 4、构建镜像 root@master1:~# bash build-command.sh查看本地镜像已经构建完成
查看是否上传到了harbor服务器上
至此就把nginx的base基础镜像构建完成了，下面再基于这个base镜像，进行nginx的业务镜像的构建
2、构建nginx业务镜像，作为前端的web服务镜像
构建业务镜像
1、看一下dockerfile同级需要的文件 root@master1:/opt/k8s-data/dockerfile/web/magedu/nginx# ls app1.tar.gz build-command.sh Dockerfile index.html nginx.conf webapp 2、编写dockerfile文件 root@master1:/opt/k8s-data/dockerfile/web/magedu/nginx# cat Dockerfile #Nginx 1.22.0 FROM harbor.magedu.net/pub-images/nginx-base:v1.22.0 ADD nginx.conf /usr/local/nginx/conf/nginx.conf #把代码放到了webapp ADD app1.tar.gz /usr/local/nginx/html/webapp/ ADD index.html /usr/local/nginx/html/index.html #静态资源挂载路径 RUN mkdir -p /usr/local/nginx/html/webapp/static /usr/local/nginx/html/webapp/images EXPOSE 80 443 CMD ["nginx"] #最后通过nginx命令把它跑起来，之所以能跑起来是因为在nginx.conf配置文件中加了参数daemon off把终端从后台关掉，从前台执行，不然nginx起不来 3、查看下首页文件 root@master1:/opt/k8s-data/dockerfile/web/magedu/nginx# cat index.html nginx web1 magedu n70 v1 4、查看下webapp/index.html文件，是通过把它打包成app1.tar.gz再传递到容器中 root@master1:/opt/k8s-data/dockerfile/web/magedu/nginx# cat webapp/index.html <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <title>Devops</title> </head> <body> <h1>magedu devops v11111111</h1> </body> </html> 5、解包app1.tar.gz看下文件内容，就是webapp/index.html文件打包而成的 root@master1:/opt/k8s-data/dockerfile/web/magedu/nginx# tar xvf app1.tar.gz -C /tmp/ index.html root@master1:/opt/k8s-data/dockerfile/web/magedu/nginx# cat /tmp/index.html <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <title>Devops</title> </head> <body> <h1>magedu devops v11111111</h1> </body> </html>nginx.conf配置文件添加下面内容，主要是用于观看，具体详细可看下面的文件
查看下nginx.conf配置文件
root@master1:/opt/k8s-data/dockerfile/web/magedu/nginx# cat nginx.conf
user nginx nginx;
worker_processes auto;
#error_log logs/error.log;
#error_log logs/error.log notice;
#error_log logs/error.log info;
#pid logs/nginx.pid;
daemon off;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
#log_format main '$remote_addr - $remote_user [$time_local] "$request" '
# '$status $body_bytes_sent "$http_referer" '
# '"$http_user_agent" "$http_x_forwarded_for"';
#access_log logs/access.log main;
sendfile on;
#tcp_nopush on;
#keepalive_timeout 0;
keepalive_timeout 65;
#gzip on;
upstream tomcat_webserver {
server tomcat-service:80;
}
server {
listen 80;
server_name localhost;
#charset koi8-r;
#access_log logs/host.access.log main;
location / {
root html;
index index.html index.htm;
}
location /webapp {
root html;
index index.html index.htm;
}
location /myapp {
proxy_pass http://tomcat_webserver;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
}
#error_page 404 /404.html;
# redirect server error pages to the static page /50x.html
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
# proxy the PHP scripts to Apache listening on 127.0.0.1:80
#
#location ~ \.php$ {
# proxy_pass http://127.0.0.1;
#}
# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
#
#location ~ \.php$ {
# root html;
# fastcgi_pass 127.0.0.1:9000;
# fastcgi_index index.php;
# fastcgi_param SCRIPT_FILENAME /scripts$fastcgi_script_name;
# include fastcgi_params;
#}
# deny access to .htaccess files, if Apache's document root
# concurs with nginx's one
#
#location ~ /\.ht {
# deny all;
#}
}
# another virtual host using mix of IP-, name-, and port-based configuration
#
#server {
# listen 8000;
# listen somename:8080;
# server_name somename alias another.alias;
# location / {
# root html;
# index index.html index.htm;
# }
#}
# HTTPS server
#
#server {
# listen 443 ssl;
# server_name localhost;
# ssl_certificate cert.pem;
# ssl_certificate_key cert.key;
# ssl_session_cache shared:SSL:1m;
# ssl_session_timeout 5m;
# ssl_ciphers HIGH:!aNULL:!MD5;
# ssl_prefer_server_ciphers on;
# location / {
# root html;
# index index.html index.htm;
# }
#}
}
3、构建业务镜像
查看构建镜像脚本
root@master1:/opt/k8s-data/dockerfile/web/magedu/nginx# cat build-command.sh
#!/bin/bash
TAG=$1
nerdctl build -t harbor.magedu.net/magedu/nginx-web1:${TAG} .
nerdctl push harbor.magedu.net/magedu/nginx-web1:${TAG}
执行脚本构建镜像，并上传到harbor
root@master1:/opt/k8s-data/dockerfile/web/magedu/nginx# bash build-command.sh v24、查看镜像
在本地查看镜像是否构建完成
在harbor服务器端查看镜像是否上传成功
5、创建nginx前端服务
5.1、编写yaml文件
root@master1:/opt/k8s-data/yaml/magedu/nginx# cat nginx.yaml
kind: Deployment
apiVersion: apps/v1
metadata:
labels:
app: nginx-deployment-label
name: nginx-deployment
namespace: myserver
spec:
replicas: 1
selector:
matchLabels:
app: nginx-selector
template:
metadata:
labels:
app: nginx-selector
spec:
containers:
- name: nginx-container
image: harbor.magedu.net/magedu/nginx-web1:v2
imagePullPolicy: Always
ports:
- containerPort: 80
protocol: TCP
name: http
- containerPort: 443
protocol: TCP
name: https
env:
- name: "password"
value: "123456"
- name: "age"
value: "20"
resources:
limits:
cpu: 500m
memory: 512Mi
requests:
cpu: 500m
memory: 256Mi
volumeMounts:
- name: nginx-images
mountPath: /usr/local/nginx/html/webapp/images
readOnly: false
- name: nginx-static
mountPath: /usr/local/nginx/html/webapp/static
readOnly: false
volumes:
- name: nginx-images
nfs:
server: 172.31.7.109
path: /data/k8sdata/images
- name: nginx-static
nfs:
server: 172.31.7.109
path: /data/k8sdata/static
---
kind: Service
apiVersion: v1
metadata:
labels:
app: nginx-service-label
name: nginx-service
namespace: myserver
spec:
type: NodePort
ports:
- name: http
port: 80
protocol: TCP
targetPort: 80
nodePort: 30090
- name: https
port: 443
protocol: TCP
targetPort: 443
nodePort: 30091
selector:
app: nginx-selector
5.2、创建资源
root@master1:/opt/k8s-data/yaml/magedu/nginx# kubectl apply -f nginx.yaml查看pod资源和service资源是否启动成功
6、访问测试
访问下node节点的30090端口是否通，可以正常访问就行
可选项：可以把访问放到负载均衡器上
把三个node节点上的30090端口的访问，放入到负载均衡VIP7.189的80端口上
重启haproxy服务后访问下VIP80端口即可访问到服务
访问webapp是由nginx处理的
访问myapp是由tomcat处理的，一访问myapp就由nginx转发到tomcat了
进入tomcat容器下载个图片
容器中有两个挂载images是用来存放图片的
进入容器的/usr/local/nginx/html/webapp/images目录下载个图片 [root@tomcat-deployment-686bdd6c68-5dmbb ~]# cd /usr/local/nginx/html/webapp/images [root@tomcat-deployment-686bdd6c68-5dmbb images]# wget http://i0.hdslb.com/bfs/banner/bda2dea2ee9cff46a7a855276f99f12b22b7393d.png [root@tomcat-deployment-686bdd6c68-5dmbb images]# ls bda2dea2ee9cff46a7a855276f99f12b22b7393d.png 把名字该短点 [root@tomcat-deployment-686bdd6c68-5dmbb images]# mv bda2dea2ee9cff46a7a855276f99f12b22b7393d.png 222.png [root@tomcat-deployment-686bdd6c68-5dmbb images]# ls 222.png这个图片其实最终是存储在存储服务器的共享目录下
并且这个其实是由tomcat写进存储的，但是nignx前端是可以访问
访问http://172.31.7.189/webapp/images/222.jpg看看能否访问到这个图片资源
433
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
