Docker环境部署，SpringCloud全家桶Docker搭建、Elasticsearch、kibana、Nacos、Sentinel、seata、skywalking、redis集群搭建

docker部署环境命令

• Elasticsearch

  • 启动命令

    docker run -d \
    -e ES_JAVA_OPTS="-Xms512m -Xmx1024m" \
    -e "discovery.type=single-node" \
    -p 9200:9200 \
    --net mynet --ip 172.18.1.4 \
    --name elasticsearch \
    elasticsearch:7.17.3
    

  • 安装分词器

    进入到docker的bin目录

    elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v7.17.3/elasticsearch-analysis-ik-7.17.3.zip
    

• kibana

  • 启动时先查看elasticsearch的容器ip

    先查看el在docker里的ip地址 docker inspect el的Id

    docker run --name kibana -e ELASTICSEARCH_HOSTS=http://172.18.1.4:9200 -p 5601:5601 --net mynet --ip 172.18.1.5 -d kibana:7.17.3
    

  • 如果提示vim not found 则以管理员身份进入/bin/bash

    (Docker exec -it -u root 容器Id /bin/bash)

    apt-get update #更新apt-get
    apt-get install vim  #下载vim
    

  • 改变config里的yml文件 设置ip和中文

    server.host: "0.0.0.0"
    server.shutdownTimeout: "5s"
    elasticsearch.hosts: [ "http://172.18.1.4:9200" ]
    monitoring.ui.container.elasticsearch.enabled: true
    i18n.locale: zh-CN
    

• Nacos

  拉取镜像

  docker pull nacos/nacos-server:1.4.1
  其他版本如果not found  在版本号前加v 例如v1.4.3
  

  • 单机

    • 启动命令（如果不使用mysql则不需要添加mysql环境）

      docker run -d \
      -e MODE=standalone \
      -e PREFER_HOST_MODE=ip \
      -e SPRING_DATASOURCE_PLATFORM=mysql \
      -e MYSQL_SERVICE_HOST=124.71.204.199 \
      -e MYSQL_SERVICE_PORT=3306 \
      -e MYSQL_SERVICE_USER=root \
      -e MYSQL_SERVICE_PASSWORD=Wtong1970. \
      -e MYSQL_SERVICE_DB_NAME=nacos \
      -e JVM_XMS=128m \
      -e JVM_XMX=256m \
      -e JVM_XMN=128m \
      -p 8848:8848 \
      --name nacos \
      --privileged=true \
      --restart=always \
      nacos/nacos-server:1.4.1
      

    • 教程网址

      点击跳转到教程地址

  • 集群

    • 启动命令（修改name和mysql地址）

      docker run -itd \
      -e PREFER_HOST_MODE=ip \
      -e MODE=cluster \
      -e NACOS_SERVERS="172.18.1.2:8848 172.18.1.3" \
      -e SPRING_DATASOURCE_PLATFORM=mysql \
      -e MYSQL_SERVICE_HOST=43.138.13.43 \
      -e MYSQL_SERVICE_PORT=3306 \
      -e MYSQL_SERVICE_DB_NAME=nacos \
      -e MYSQL_SERVICE_USER=root \
      -e MYSQL_SERVICE_PASSWORD=root \
      -e JVM_XMS=128m \
      -e JVM_XMX=256m \
      -e JVM_XMN=128m \
      -p 8849:8848 \
      --name nacos01 \
      --net mynet --ip 172.18.1.1 \
      --privileged=true \
      --restart=always \
      nacos/nacos-server:v1.4.3
      

      注：

      • –net 后边的网络要用docker里创建过的网络 可以使用 docker network ls 查看，否则会出现not found错误

      • docker: Error response from daemon: user specified IP address is supported on user defined networks only.

      ​ 出现这种错误是因为docker中，只有使用 --subnet创建的网络才能指定静态IP

      docker network create --driver bridge --subnet 172.18.1.0/16 --gateway 172.18.1.0 mynet
      

• Sentinel

  • 下载地址

    docker pull bladex/sentinel-dashboard:1.8.0
    

  • 启动

    docker run  -d --name sentinel \
    -p 8858:8858 \
    bladex/sentinel-dashboard:1.8.0
    

  • 注意：

    • 如果sentinel部署在远程服务器，本地运行客户端会接收不到请求数据以及配置信息，猜测可能是因为本地的8719端口没有开放，所以远程访问不到

• seata

  • 下载地址

    docker pull seataio/seata-server:1.3.0
    

  • 数据源设置

    ## transaction log store, only used in seata-server
    store {
      ## store mode: file、db、redis
      mode = "db"
    
      ## file store property
      file {
        ## store location dir
        dir = "sessionStore"
        # branch session size , if exceeded first try compress lockkey, still exceeded throws exceptions
        maxBranchSessionSize = 16384
        # globe session size , if exceeded throws exceptions
        maxGlobalSessionSize = 512
        # file buffer size , if exceeded allocate new buffer
        fileWriteBufferCacheSize = 16384
        # when recover batch read size
        sessionReloadReadSize = 100
        # async, sync
        flushDiskMode = async
      }
    
      ## database store property
      db {
        ## the implement of javax.sql.DataSource, such as DruidDataSource(druid)/BasicDataSource(dbcp)/HikariDataSource(hikari) etc.
        datasource = "druid"
        ## mysql/oracle/postgresql/h2/oceanbase etc.
        dbType = "mysql"
        driverClassName = "com.mysql.cj.jdbc.Driver"
        url = "jdbc:mysql://43.138.13.43:3306/seata"
        user = "root"
        password = "root"
        minConn = 5
        maxConn = 30
        globalTable = "global_table"
        branchTable = "branch_table"
        lockTable = "lock_table"
        queryLimit = 100
        maxWait = 5000
      }
    
      ## redis store property
      redis {
        host = "127.0.0.1"
        port = "6379"
        password = ""
        database = "0"
        minConn = 1
        maxConn = 10
        queryLimit = 100
      }
    }
    
    

    注意：

    • 修改mode=“db” 修改数据库相关信息

  • 注册中心配置中心设置

    registry {
      # file 、nacos 、eureka、redis、zk、consul、etcd3、sofa
      type = "nacos"
      loadBalance = "RandomLoadBalance"
    
      nacos {
        application = "seata-server"
        serverAddr = "43.138.13.43:8848"
        group = "SEATA_GROUP"
        namespace = ""   # 用来做资源隔离的
        cluster = "default"
        username = "nacos"
        password = "nacos"
      }
      
    }
    
    config {
      # file、nacos 、apollo、zk、consul、etcd3、springCloudConfig
      type = "nacos"
    
      nacos {
        serverAddr = "43.138.13.43:8848"
        namespace = ""
        group = "SEATA_GROUP"
        cluster = "default"
        username = "nacos"
        password = "nacos"
      }
     
    }
    
    
    

  • ​ config.txt

    #For details about configuration items, see https://seata.io/zh-cn/docs/user/configurations.html
    #Transport configuration, for client and server
    transport.type=TCP
    transport.server=NIO
    transport.heartbeat=true
    transport.enableTmClientBatchSendRequest=false
    transport.enableRmClientBatchSendRequest=true
    transport.enableTcServerBatchSendResponse=false
    transport.rpcRmRequestTimeout=30000
    transport.rpcTmRequestTimeout=30000
    transport.rpcTcRequestTimeout=30000
    transport.threadFactory.bossThreadPrefix=NettyBoss
    transport.threadFactory.workerThreadPrefix=NettyServerNIOWorker
    transport.threadFactory.serverExecutorThreadPrefix=NettyServerBizHandler
    transport.threadFactory.shareBossWorker=false
    transport.threadFactory.clientSelectorThreadPrefix=NettyClientSelector
    transport.threadFactory.clientSelectorThreadSize=1
    transport.threadFactory.clientWorkerThreadPrefix=NettyClientWorkerThread
    transport.threadFactory.bossThreadSize=1
    transport.threadFactory.workerThreadSize=default
    transport.shutdown.wait=3
    transport.serialization=seata
    transport.compressor=none
    
    #Transaction routing rules configuration, only for the client
    service.vgroupMapping.default_tx_group=default
    #If you use a registry, you can ignore it
    service.default.grouplist=127.0.0.1:8091
    service.enableDegrade=false
    service.disableGlobalTransaction=false
    
    #Transaction rule configuration, only for the client
    client.rm.asyncCommitBufferLimit=10000
    client.rm.lock.retryInterval=10
    client.rm.lock.retryTimes=30
    client.rm.lock.retryPolicyBranchRollbackOnConflict=true
    client.rm.reportRetryCount=5
    client.rm.tableMetaCheckEnable=true
    client.rm.tableMetaCheckerInterval=60000
    client.rm.sqlParserType=druid
    client.rm.reportSuccessEnable=false
    client.rm.sagaBranchRegisterEnable=false
    client.rm.sagaJsonParser=fastjson
    client.rm.tccActionInterceptorOrder=-2147482648
    client.tm.commitRetryCount=5
    client.tm.rollbackRetryCount=5
    client.tm.defaultGlobalTransactionTimeout=60000
    client.tm.degradeCheck=false
    client.tm.degradeCheckAllowTimes=10
    client.tm.degradeCheckPeriod=2000
    client.tm.interceptorOrder=-2147482648
    client.undo.dataValidation=true
    client.undo.logSerialization=jackson
    client.undo.onlyCareUpdateColumns=true
    server.undo.logSaveDays=7
    server.undo.logDeletePeriod=86400000
    client.undo.logTable=undo_log
    client.undo.compress.enable=true
    client.undo.compress.type=zip
    client.undo.compress.threshold=64k
    #For TCC transaction mode
    tcc.fence.logTableName=tcc_fence_log
    tcc.fence.cleanPeriod=1h
    
    #Log rule configuration, for client and server
    log.exceptionRate=100
    
    #Transaction storage configuration, only for the server. The file, db, and redis configuration values are optional.
    store.mode=db
    store.lock.mode=file
    store.session.mode=file
    #Used for password encryption
    store.publicKey=
    
    #If `store.mode,store.lock.mode,store.session.mode` are not equal to `file`, you can remove the configuration block.
    store.file.dir=file_store/data
    store.file.maxBranchSessionSize=16384
    store.file.maxGlobalSessionSize=512
    store.file.fileWriteBufferCacheSize=16384
    store.file.flushDiskMode=async
    store.file.sessionReloadReadSize=100
    
    #These configurations are required if the `store mode` is `db`. If `store.mode,store.lock.mode,store.session.mode` are not equal to `db`, you can remove the configuration block.
    store.db.datasource=druid
    store.db.dbType=mysql
    store.db.driverClassName=com.mysql.cj.jdbc.Driver
    store.db.url=jdbc:mysql://43.138.13.43:3306/seata?useUnicode=true&serverTimezone=Asia/Shanghai
    store.db.user=root
    store.db.password=root
    store.db.minConn=5
    store.db.maxConn=30
    store.db.globalTable=global_table
    store.db.branchTable=branch_table
    store.db.distributedLockTable=distributed_lock
    store.db.queryLimit=100
    store.db.lockTable=lock_table
    store.db.maxWait=5000
    
    #These configurations are required if the `store mode` is `redis`. If `store.mode,store.lock.mode,store.session.mode` are not equal to `redis`, you can remove the configuration block.
    store.redis.mode=single
    store.redis.single.host=127.0.0.1
    store.redis.single.port=6379
    store.redis.sentinel.masterName=
    store.redis.sentinel.sentinelHosts=
    store.redis.maxConn=10
    store.redis.minConn=1
    store.redis.maxTotal=100
    store.redis.database=0
    store.redis.password=
    store.redis.queryLimit=100
    
    #Transaction rule configuration, only for the server
    server.recovery.committingRetryPeriod=1000
    server.recovery.asynCommittingRetryPeriod=1000
    server.recovery.rollbackingRetryPeriod=1000
    server.recovery.timeoutRetryPeriod=1000
    server.maxCommitRetryTimeout=-1
    server.maxRollbackRetryTimeout=-1
    server.rollbackRetryTimeoutUnlockEnable=false
    server.distributedLockExpireTime=10000
    server.xaerNotaRetryTimeout=60000
    server.session.branchAsyncQueueSize=5000
    server.session.enableBranchAsyncRemove=false
    server.enableParallelRequestHandle=false
    
    #Metrics configuration, only for the server
    metrics.enabled=false
    metrics.registryType=compact
    metrics.exporterList=prometheus
    metrics.exporterPrometheusPort=9898
    

    注意：

    • 修改db.mode=db 还有数据库相关信息
    • 注意使用docker部署nacos会有内网ip，所以不要使用127.0.0.1，要使用宿主机ip或docker容器内部ip

  • nacos-config.sh

    #!/bin/sh
    # Copyright 1999-2019 Seata.io Group.
    #
    # Licensed under the Apache License, Version 2.0 (the "License");
    # you may not use this file except in compliance with the License.
    # You may obtain a copy of the License at、
    #
    #      http://www.apache.org/licenses/LICENSE-2.0
    #
    # Unless required by applicable law or agreed to in writing, software
    # distributed under the License is distributed on an "AS IS" BASIS,
    # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    # See the License for the specific language governing permissions and
    # limitations under the License.
    
    while getopts ":h:p:g:t:u:w:" opt
    do
      case $opt in
      h)
        host=$OPTARG
        ;;
      p)
        port=$OPTARG
        ;;
      g)
        group=$OPTARG
        ;;
      t)
        tenant=$OPTARG
        ;;
      u)
        username=$OPTARG
        ;;
      w)
        password=$OPTARG
        ;;
      ?)
        echo " USAGE OPTION: $0 [-h host] [-p port] [-g group] [-t tenant] [-u username] [-w password] "
        exit 1
        ;;
      esac
    done
    
    if [ -z ${host} ]; then
        host=localhost
    fi
    if [ -z ${port} ]; then
        port=8848
    fi
    if [ -z ${group} ]; then
        group="SEATA_GROUP"
    fi
    if [ -z ${tenant} ]; then
        tenant=""
    fi
    if [ -z ${username} ]; then
        username=""
    fi
    if [ -z ${password} ]; then
        password=""
    fi
    
    nacosAddr=$host:$port
    contentType="content-type:application/json;charset=UTF-8"
    
    echo "set nacosAddr=$nacosAddr"
    echo "set group=$group"
    
    urlencode() {
      length="${#1}"
      i=0
      while [ $length -gt $i ]; do
        char="${1:$i:1}"
        case $char in
        [a-zA-Z0-9.~_-]) printf $char ;;
        *) printf '%%%02X' "'$char" ;;
        esac
        i=`expr $i + 1`
      done
    }
    
    failCount=0
    tempLog=$(mktemp -u)
    function addConfig() {
      dataId=`urlencode $1`
      content=`urlencode $2`
      curl -X POST -H "${contentType}" "http://$nacosAddr/nacos/v1/cs/configs?dataId=$dataId&group=$group&content=$content&tenant=$tenant&username=$username&password=$password" >"${tempLog}" 2>/dev/null
      if [ -z $(cat "${tempLog}") ]; then
        echo " Please check the cluster status. "
        exit 1
      fi
      if [ "$(cat "${tempLog}")" == "true" ]; then
        echo "Set $1=$2 successfully "
      else
        echo "Set $1=$2 failure "
        failCount=`expr $failCount + 1`
      fi
    }
    
    count=0
    COMMENT_START="#"
    for line in $(cat $(dirname "$PWD")/config.txt | sed s/[[:space:]]//g); do
        if [[ "$line" =~ ^"${COMMENT_START}".*  ]]; then
          continue
        fi
        count=`expr $count + 1`
    	  key=${line%%=*}
        value=${line#*=}
    	  addConfig "${key}" "${value}"
    done
    
    echo "========================================================================="
    echo " Complete initialization parameters,  total-count:$count ,  failure-count:$failCount "
    echo "========================================================================="
    
    if [ ${failCount} -eq 0 ]; then
    	echo " Init nacos config finished, please start seata-server. "
    else
    	echo " init nacos config fail. "
    fi
    

    将配置信息存储到nacos脚本

    sh nacos-config.sh -h 43.138.13.43 -p 8848 -g SEATA_GROUP -u nacos -w nacos
    

    注意：

    • 不加-u -w 会倒入失败 ，注意查看是否导入成功，如果导入失败后续运行容器会报错

  • 运行容器

    docker run -itd --name seata-server \
    -p 8091:8091 \
    -e SEATA_CONFIG_NAME=file:/root/seata-config/registry \
    -e SEATA_IP=43.138.13.43 \
    -e STORE_MODE=db \
    -v /usr/docker/seata/config:/root/seata-config  \
    seataio/seata-server:1.3.0
    

  • 开放端口，如果是云服务器需要在云服务器也开放对应端口

  • skywalking

    • 安装（注意版本对应）

      docker pull apache/skywalking-oap-server:8.9.1
      docker pull apache/skywalking-ui:8.9.1
      

    • 运行Skywalking OAP

      docker run --name oap -d \
      -e TZ=Asia/Shanghai \
      -p 12800:12800 \
      -p 11800:11800 \
      -e SW_STORAGE=elasticsearch \
      -e SW_STORAGE_ES_CLUSTER_NODES=172.18.1.4:9200 \
      -v /etc/localtime:/etc/localtime:ro \
      --net mynet --ip 172.18.1.6 \
      apache/skywalking-oap-server:8.9.1
      

    • 运行Skywalking UI

      docker run \
      --name oap-ui \
      -p 8088:8080 -d \
      --privileged=true \
      --link skywalking-oap:skywalking-oap \
      -e TZ=Asia/Shanghai \
      -e SW_OAP_ADDRESS=http://172.18.1.6:12800 \
      -v /etc/localtime:/etc/localtime:ro \
      --net mynet --ip 172.18.1.7 \
      apache/skywalking-ui:8.9.1
      
      docker run --name oap-ui -d -e SW_OAP_ADDRESS=http://oap:12800 apache/skywalking-ui
      

    • ​ 适配getway待实现

    • 修改报警配置文件

      1. 进入容器内部

      2. 下载vim

         linux系统版本是Alpine
         下载命令
         apk search openssh  #查询openssh相关的软件包
         apk add xxx  #安装一个软件包
         apk del xxx  #删除已安装的xxx软件包
         apk --help  #获取更多apk包管理的命令参数
         apk update   #更新软件包索引文件
         

      3. 修改config/alarm-settings.yml

         修改webhooks下的地址为回掉地址 注意以yum数组的方式设置

         webhooks:
          - http.......
         

  • redis集群搭建

    #通过脚本一次创建6个redis配置
    for port in $(seq 1 6); \
    do \
    mkdir -p /mydata/redis/node-${port}/conf
    touch /mydata/redis/node-${port}/conf/redis.conf
    cat << EOF >/mydata/redis/node-${port}/conf/redis.conf
    port 6379
    bind 0.0.0.0
    masterauth Wtong1970.
    requirepass Wtong1970.
    cluster-enabled yes
    cluster-config-file nodes.conf
    cluster-node-timeout 5000
    cluster-announce-ip 172.38.0.1${port}
    cluster-announce-port 6379
    cluster-announce-bus-port 16379
    appendonly yes
    EOF
    # 通过脚本一次启动6个redis容器
    docker run -p 637${port}:6379 -p 1637${port}:16379 --name redis-${port} \
    -v /mydata/redis/node-${port}/data:/data \
    -v /mydata/redis/node-${port}/conf/redis.conf:/etc/redis/redis.conf \
    -d --net redis --ip 172.38.0.1${port} redis redis-server /etc/redis/redis.conf
    done
    

    更改redis版本

    之后执行

    docker exec -it redis-1 /bin/sh
    

    进入redis-1的容器内之后执行

    redis-cli -a Wtong1970. --cluster create 172.38.0.11:6379 172.38.0.12:6379 172.38.0.13:6379 172.38.0.14:6379 172.38.0.15:6379 172.38.0.16:6379 --cluster-replicas 1
    

    之后可以通过

    redis-cli -c
    

    进入集群内部

    cluster nodes
    

    可以查看node

    注意： 如果外网需要外网连接 则需要把内网地址全换成外网可以访问到的地址，并且将redis保护关闭，将6371-6376端口和16371-16376端口全部开放

    例如

    #通过脚本一次创建6个redis配置
    for port in $(seq 1 6); \
    do \
    mkdir -p /mydata/redis/node-${port}/conf
    touch /mydata/redis/node-${port}/conf/redis.conf
    cat << EOF >/mydata/redis/node-${port}/conf/redis.conf
    port 637${port}
    bind 0.0.0.0
    Protected-mode no
    masterauth Wtong1970.
    requirepass Wtong1970.
    cluster-enabled yes
    cluster-config-file nodes.conf
    cluster-node-timeout 5000
    cluster-announce-ip 124.71.204.199
    cluster-announce-port 637${port}
    cluster-announce-bus-port 1637${port}
    appendonly yes
    EOF
    # 通过脚本一次启动6个redis容器
    docker run -p 637${port}:6379 -p 1637${port}:16379 --name redis-${port} \
    -v /mydata/redis/node-${port}/data:/data \
    -v /mydata/redis/node-${port}/conf/redis.conf:/etc/redis/redis.conf \
    -d --net redis --ip 172.38.0.1${port} redis redis-server /etc/redis/redis.conf
    done
    

    创建好conf和容器之后执行下边

    docker exec -it redis-1 /bin/sh
    

    进入随意一个容器中再执行cluster连接

    redis-cli -a Wtong1970. --cluster create 124.71.204.199:6371 124.71.204.199:6372 124.71.204.199:6373 124.71.204.199:6374 124.71.204.199:6375 124.71.204.199:6376 --cluster-replicas 1