什么是CI/CD，什么是Iac，一篇文章带你从入门到实战自动化部署项目

Dlihctcefrep

于 2024-05-01 07:50:09 发布

阅读量410

点赞数 3

文章标签：云计算 ci/cd terraform IaC

本文链接：https://blog.csdn.net/Dlihctcefrep/article/details/138365084

版权

前言，很久没有写文章了。前端时间学习了Infrastructure as Code相关的知识，希望以此文总结一下所学的知识。

1. 概念

1.1 什么是CI/CD，为什么需要CI/CD？

当我们开发完一个功能模块想要部署上线的时候，如果没有持续集成/部署，那么你就需要自己运行测试，并打包部署到服务器。如果服务器只有一台，也许你还能轻松应对。但是当服务器有很多的时候，估计你就汗流浃背了。
在这里插入图片描述

这个时候，我们就需要使用CI/CD来实现新功能自动测试以及上线的功能。
例如，我们可以编写对应的单元测试和集成测试，在提交PR的时候，GitHub运行action来执行对应的测试也就是Continuous Integration。
对于Continuous Deployment这一部分，在PR合并到主分支后，可以运行Packer来将仓库代码生成对应的jar包，并部署到服务器上。后面会详细说明

1.2 什么是IaC，为什么IaC？

IaC是基础设施即代码的缩写，也是DevOps中必不可少的一部分。想必大家都有手动配置云服务器的经历吧，有时候配置简单的资源的时候还可以勉强应付。但是当需要配置复杂的资源以及配置海量资源时，手动配置就不是一个很明智的选择。毕竟人不是机器，有时候会忘掉配置防火墙，Vpc，又或者是配置的时候输入错了某些参数，又或者是需要你配置成百上千台服务器的时候，总不可能都用手动配置吧。

这个时候，我们就需要使用IaC来帮助我们一劳永逸。我们可以使用代码，来告诉云服务商AWS，GCP来为我创建什么资源，并且再创建以后，也可以一键销毁，大大提高了开发的效率。常见的配置和管理云基础架构和资源的服务商有Terraform

2. 自动化部署实战

在这里插入图片描述
对于一个简单的应用，我们需要一个服务器，一个数据库，数据库只能由服务器访问。我们只需要用terraform创建一个VM，一个DB，对应的防火墙和VPC即可。Terraform的代码在文章末尾。

这个时候，我们可以使用terraform apply来创建对应的云服务器资源。每次当我们添加了新的功能以后，在github merge了PR。我们就可以执行相应的github action来帮助我们给程序打包，并替换掉当前在线上的服务器，从而实现自动化部署的流程。对应的action文件如下：

# This workflow will compile a package using Maven and then publish it to GitHub packages when a release is created
name: Build machine image

on:
  pull_request:
    types: [closed]
    branches:
      - main

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3
      - name: Set up JDK 8
        uses: actions/setup-java@v3
        with:
          java-version: '8'
          distribution: 'temurin'

      - uses: shogo82148/actions-setup-mysql@v1
        with:
          mysql-version: "8.0"
          root-password: ${{ secrets.MYSQL_ROOT_PASSWORD }}

      - id: auth
        uses: google-github-actions/auth@v2
        with:
          credentials_json: ${{ secrets.GCP_ACCOUNT_JSON }}

      - name: Set up Cloud SDK
        uses: google-github-actions/setup-gcloud@v2

      - name: Use gcloud CLI
        run: gcloud info

      - name: Run integration tests
        run: mvn -B clean test -Djasypt.encryptor.password=${{ secrets.JASYPT_ENCRYPTION_KEY }} -Dlogback.log.path="."

      - name: Install Package
        run: mvn -B clean install -DskipTests -Djasypt.encryptor.password=${{ secrets.JASYPT_ENCRYPTION_KEY }} -Dlogback.log.path="/var/log"

      - name: Setup Packer
        uses: hashicorp/setup-packer@main
        id: setup
        with:
          version: 1.10.1

      - name: Run `packer init`
        id: init
        run: "packer init ./gcp-centos8.pkr.hcl"

      - name: Build Image
        run: |
          packer build -on-error=abort -var 'project_id='${{ secrets.GCP_PROJECT_ID }} \
          -var 'jasypt_encryption_key='${{ secrets.JASYPT_ENCRYPTION_KEY }} \
          -var 'zone='${{ secrets.GCP_ZONE }} ./gcp-centos8.pkr.hcl | tee packer_output.txt

      - name: Extract Image ID
        id: extract_image_id
        run: |
          IMAGE_ID=$(tail -2 packer_output.txt | awk 'match($0, /packer-.*/) { print substr($0, RSTART, RLENGTH) }')
          echo "IMAGE_ID=${IMAGE_ID}" >> $GITHUB_ENV

      - name: Create Startup Scripts
        run: |
          echo "${{ secrets.STARTUP_SCRIPT }}" | base64 -d > startup-script.sh

      # New Steps for Creating and Updating Instance Template
      - name: Create New Instance Template Version
        run: |
          gcloud compute instance-templates create "instance-template-${{ github.run_id }}-${{ github.run_attempt }}" \
            --region=${{ secrets.GCP_REGION }} \
            --instance-template-region=${{ secrets.GCP_REGION }} \
            --machine-type=${{ vars.MACHINE_TYPE }} \
            --tags="webapp-${{ vars.RANDOM_SUFFIX }}",allow-health-check \
            --create-disk=auto-delete=true,size=${{ vars.DISK_SIZE }},type=${{ vars.DISK_TYPE }},image=${IMAGE_ID},boot=${{ vars.IS_BOOT }},kms-key="projects/${{ secrets.GCP_PROJECT_ID }}/locations/${{ secrets.GCP_REGION }}/keyRings/webapp-key-ring-${{ vars.RANDOM_SUFFIX }}/cryptoKeys/vm-key" \
            --network-interface=network="vpc-network-${{ vars.RANDOM_SUFFIX }}",subnet="webapp-subnet-${{ vars.RANDOM_SUFFIX }}",no-address \
            --metadata-from-file=startup-script=./startup-script.sh \
            --service-account=${{ secrets.VM_SERVICE_ACCOUNT }} \
            --scopes=cloud-platform

      - name: Configure Managed Instance Group
        run: |
          gcloud compute instance-groups managed set-instance-template instance-group-manager \
            --template="projects/${{ secrets.GCP_PROJECT_ID }}/regions/${{ secrets.GCP_REGION }}/instanceTemplates/instance-template-${{ github.run_id }}-${{ github.run_attempt }}" \
            --region=${{ secrets.GCP_REGION }}

      - name: Recreate Instances in Managed Instance Group, and wait for updates to complete
        run: |
          gcloud compute instance-groups managed rolling-action start-update instance-group-manager \
          --region=${{ secrets.GCP_REGION }} --version="template=projects/${{ secrets.GCP_PROJECT_ID }}/regions/${{ secrets.GCP_REGION }}/instanceTemplates/instance-template-${{ github.run_id }}-${{ github.run_attempt }}"
          while :; do
            STATUS=$(gcloud compute instance-groups managed describe instance-group-manager \
              --region=${{ secrets.GCP_REGION }} --format="get(status.versionTarget.isReached)")
            echo "Current group status: $STATUS"
            if [ "$STATUS" = "True" ]; then
              echo "The Instance Group Manager has completed the refresh process"
              break
            else
              echo "The Instance Group Manager has not completed the refresh process, please wait for that......"
              sleep 10
            fi
          done

# Google Cloud Platform Provider
provider "google" {
  project = var.project_id
  region  = var.region
}

resource "random_string" "name_suffix" {
  length  = 6
  special = false
  lower   = true
  upper   = false
}

resource "google_compute_network" "vpc_network" {
  count                           = var.vpc_count
  name                            = "vpc-network-${count.index + 1}-${random_string.name_suffix.result}"
  auto_create_subnetworks         = false
  delete_default_routes_on_create = true
  routing_mode                    = var.vpc_routing_mode
}

resource "google_compute_subnetwork" "webapp_subnet" {
  count                    = var.vpc_count
  name                     = "webapp-subnet-${count.index + 1}-${random_string.name_suffix.result}"
  ip_cidr_range            = var.ip_cidr_ranges[count.index * 2]
  region                   = var.region
  network                  = google_compute_network.vpc_network[count.index].id
  private_ip_google_access = true
}

# Route for webapp
resource "google_compute_route" "webapp_route" {
  count            = var.vpc_count
  name             = "webapp-route-${count.index + 1}-${random_string.name_suffix.result}"
  network          = google_compute_network.vpc_network[count.index].id
  dest_range       = var.webapp_route_dest_range
  next_hop_gateway = "default-internet-gateway"
  priority         = 1000
  tags             = ["webapp-${count.index + 1}-${random_string.name_suffix.result}"]
}

resource "google_compute_firewall" "vpc_firewall_webapp_allow_rule" {
  count    = var.vpc_count
  name     = "vpc-firewall-webapp-allow-rule-${count.index + 1}-${random_string.name_suffix.result}"
  network  = google_compute_network.vpc_network[count.index].id
  priority = var.firewall.allow_rule_priority
  allow {
    protocol = var.firewall.allow_rule_protocol
    ports    = var.firewall.allow_rule_ports
  }
  source_ranges = var.firewall.allow_rule_source_ranges
  target_tags   = ["webapp-${count.index + 1}-${random_string.name_suffix.result}"]
}

// Explicit to deny all another rule as required
resource "google_compute_firewall" "vpc_firewall_deny_rule" {
  count    = var.vpc_count
  name     = "vpc-firewall-deny-rule-${count.index + 1}-${random_string.name_suffix.result}"
  network  = google_compute_network.vpc_network[count.index].id
  priority = var.firewall.deny_rule_priority
  deny {
    protocol = var.firewall.deny_rule_protocol
  }
  source_ranges = var.firewall.deny_rule_source_ranges
  target_tags   = ["webapp-${count.index + 1}-${random_string.name_suffix.result}"]
}

resource "google_compute_instance" "vm_instance" {
  count        = var.vpc_count
  name         = "vm-instance-${count.index + 1}-${random_string.name_suffix.result}"
  machine_type = var.vm_machine_type
  zone         = var.zone
  tags         = ["webapp-${count.index + 1}-${random_string.name_suffix.result}"]
  boot_disk {
    initialize_params {
      image = var.vm_boot_disk_params.image
      size  = var.vm_boot_disk_params.size
      type  = var.vm_boot_disk_params.type
    }
  }
  network_interface {
    network    = google_compute_network.vpc_network[count.index].id
    subnetwork = google_compute_subnetwork.webapp_subnet[count.index].id
    access_config {}
  }
  metadata_startup_script = <<-EOF
    #!/bin/bash
    sudo yum update -y
    echo "y" | sudo yum install -y mysql

    sudo cat <<EOT > /opt/webapp_repo/startup.sh
    #!/bin/bash
    DB_HOST="${var.psc_addrs[count.index]}"
    DB_USER="webapp"
    DB_PASSWORD=${random_password.mysql_password.result}
    java -jar /opt/webapp_repo/Health_Check-0.0.1-SNAPSHOT.jar --spring.datasource.username=\$DB_USER \
    --spring.datasource.password=\$DB_PASSWORD \
    --spring.datasource.url="jdbc:mysql://\$DB_HOST:3306/health_check?useUnicode=true&characterEncoding=utf-8&serverTimezone=America/New_York&createDatabaseIfNotExist=true" 
    EOT
    
    sudo chmod +x /opt/webapp_repo/startup.sh
    sudo systemctl daemon-reload
    sudo systemctl start webapp
  EOF
  depends_on              = [google_sql_database_instance.db_instance]
}

resource "google_compute_address" "psc_address" {
  count        = var.vpc_count
  project      = var.project_id
  name         = "psc-address-${count.index + 1}-${random_string.name_suffix.result}"
  region       = var.region
  address_type = "INTERNAL"
  subnetwork   = google_compute_subnetwork.webapp_subnet[count.index].id
  address      = var.psc_addrs[count.index]
}

resource "google_compute_forwarding_rule" "psc_endpoint" {
  count                   = var.vpc_count
  project                 = var.project_id
  name                    = "psc-endpoint-${count.index + 1}-${random_string.name_suffix.result}"
  region                  = var.region
  target                  = google_sql_database_instance.db_instance[count.index].psc_service_attachment_link
  network                 = google_compute_network.vpc_network[count.index].id
  ip_address              = google_compute_address.psc_address[count.index].id
  load_balancing_scheme   = ""
  allow_psc_global_access = true
}

resource "google_sql_database_instance" "db_instance" {
  count               = var.vpc_count
  name                = "db-instance-${count.index + 1}-${random_string.name_suffix.result}"
  region              = var.region
  database_version    = var.database_instance_config.database_version
  deletion_protection = var.database_instance_config.deletion_protection

  settings {
    tier = var.database_instance_config.settings.tier
    ip_configuration {
      ipv4_enabled                                  = false
      enable_private_path_for_google_cloud_services = true
      psc_config {
        psc_enabled               = true
        allowed_consumer_projects = [var.project_id]
      }
    }
    backup_configuration {
      enabled            = var.database_instance_config.settings.backup_configuration.enabled
      binary_log_enabled = var.database_instance_config.settings.backup_configuration.binary_log_enabled
    }
    availability_type = var.database_instance_config.settings.availability_type
    disk_type         = var.database_instance_config.settings.disk_type
    disk_size         = var.database_instance_config.settings.disk_size
    edition           = var.database_instance_config.settings.edition
  }

}

resource "google_sql_database" "database" {
  count    = var.vpc_count
  name     = "webapp-db-${count.index + 1}-${random_string.name_suffix.result}"
  instance = google_sql_database_instance.db_instance[count.index].name
}

resource "google_sql_user" "db_user" {
  count    = var.vpc_count
  name     = var.db_username
  password = random_password.mysql_password.result
  instance = google_sql_database_instance.db_instance[count.index].name
  host     = "%"
}

resource "random_password" "mysql_password" {
  length  = 16
  special = false
}