【 Azure 】基于aks-engine的kubernetes集群部署

最新推荐文章于 2024-08-12 11:26:47 发布

潘星文sama

最新推荐文章于 2024-08-12 11:26:47 发布

阅读量679

点赞数

分类专栏： azure kubernetes 文章标签： kubernetes azure docker

本文链接：https://blog.csdn.net/weixin_42526859/article/details/112241297

版权

kubernetes 同时被 2 个专栏收录

11 篇文章 3 订阅

订阅专栏

azure

4 篇文章 0 订阅

订阅专栏

【 Azure 】基于aks-engine的kubernetes集群部署

AKS Engine provides tooling to quickly bootstrap Kubernetes clusters on Azure. By leveraging ARM (Azure Resource Manager), AKS Engine helps you create, destroy and maintain clusters provisioned with basic IaaS resources in Azure. AKS Engine is limited in its support for ongoing operational needs such as scaling, in-place upgrading, and extension management. The Cluster API Provider for Azure a.k.a. CAPZ provides more complete operational capabilities. AKS Engine remains the tool for managing Kubernetes clusters on Azure Stack Hub as CAPZ does not yet work there.

AKS engine和aks的区别

AKS engine和aks都是azure中管理部署kubernetes集群的，其中AKS engine是社区开源版的，aks是产品级别的。两者都是免费的，只需要支付集群中节点费用即可。相对于aks，aks engine支持的功能更多，技术上也更超前。

AKS engine功能简介

支持Azure Active Directory(AAD)集成，即可以通过AAD进行集群认证，以及权限配置
支持extension，即可以给集群虚拟机配置extension，比如收集系统日志的omsagent
支持配置GPU
支持public/private集群，以及可配置的CNI，比如Azure CNI，flannel，cilium等
支持集成azure的keyvault
支持部署集群到windows
支持集群的弹性伸缩
支持集群VMSS Node Pools更新
支持kubernetes集群升级

部署集群

前置条件

An Azure Subscription
The Azure CLI
resource group
Vnet/Subnet

部署集群

下载aks-engine

wget https://github.com/Azure/aks-engine/releases/download/v0.55.4/aks-engine-v0.55.4-linux-amd64.tar.gz

tar -xvf aks-engine-v0.55.4-linux-amd64.tar.gz

cd aks-engine-v0.55.4-linux-amd64

配置aks engine template
template包含对azure资源的配置，以及kubernetes集群的配置，下面是个配置例子akse.json，包含了如下的需求

部署高可用的kubernetes1.17.11
private集群，外部不可访问
选用standard的loadblancer
cni选用flannel
节点启用MSI（useManagedIdentity）
安装跳板机用来访问管理集群
配置集群autoscaler
安装tiller
集成azure keyvault
集成container-monitoring来收集日志以及连接azure monitor
3 masters ，2 agent pools
开启AAD认证

akse.json

{
  "apiVersion": "vlabs",
  "location": "chinanorth2",
  "properties": {
    "orchestratorProfile": {
      "orchestratorType": "Kubernetes",
      "orchestratorRelease": "1.17",
      "orchestratorVersion": "1.17.11",
      "kubernetesConfig": {
        "loadBalancerSku": "Standard",
	"excludeMasterFromStandardLB": true,
	"kubeProxyMode" : "ipvs",
	"networkPlugin": "flannel",
    "networkPolicy": "",
    "useManagedIdentity": true,
    "userAssignedID": "/subscriptions/xxxx-xxxxx-xxxxx-xxxx/resourceGroups/xxxx-xxxxx-xxxxx-xxxx/providers/Microsoft.ManagedIdentity/userAssignedIdentities/xxxx-xxxxx-xxxxx-xxxx",    
        "privateCluster": {
          "enabled": true,
	  "jumpboxProfile": {
	     "name": "reh-corebe-jumpbox",
	     "vmSize": "Standard_B2ms",
	     "osDiskSizeGB": 30,
	     "username": "vmadmin",
	     "publickey": "ssh-rsa "
	   }
        },
	"addons": [
        {
            "name": "cluster-autoscaler",
            "enabled": true,
            "pools": [
              {
                "name": "pool1",
                "config": {
                  "min-nodes": "4",
                  "max-nodes": "500"
                }
              },
              {
                "name": "pool2",
                "config": {
                  "min-nodes": "3",
                  "max-nodes": "3"
                }
              }
            ],
            "config": {
              "scan-interval": "1m"
            }
          },
	  {
            "name": "heapster",
	    "enabled": true
          },
	  {
            "name": "dns-autoscaler",
            "enabled": false
          },
          {
            "name": "container-monitoring",
            "enabled": true,
            "config": {
              "logAnalyticsWorkspaceResourceId" :  "/subscriptions/xxxx-xxxxx-xxxxx-xxxx/resourceGroups/xxxx-xxxxx-xxxxx-xxxx/providers/Microsoft.OperationalInsights/workspaces/xxxx-xxxxx-xxxxx-xxxx"
          }
          },
          {
            "name": "tiller",
            "enabled": true,
	    "containers": [
              {
                "name": "tiller",
                "image": "gcr.azk8s.cn/kubernetes-helm/tiller:v2.13.0",
                "cpuRequests": "50m",
                "memoryRequests": "150Mi",
                "cpuLimits": "50m",
                "memoryLimits": "150Mi"
              }
            ],
            "config": {
              "max-history": "0"
            }
          },
          {
            "name": "blobfuse-flexvolume",
            "enabled": true
          },
          {
            "name": "keyvault-flexvolume",
            "enabled": true
          },
          {
            "name": "kubernetes-dashboard",
            "enabled": true
          },
          {
            "name": "aci-connector",
            "enabled": false
          },
          {
            "name": "smb-flexvolume",
            "enabled": false
          },
          {
            "name": "rescheduler",
            "enabled": false
          },
          {
            "name": "nvidia-device-plugin",
            "enabled": false
          }
        ]
       }
    },
      "masterProfile": {
        "count": 3,
        "dnsPrefix": "master",
        "vmSize": "Standard_B12ms",
	"vnetSubnetId": "/subscriptions/xxxx-xxxxx-xxxxx-xxxx/resourceGroups/xxxx-xxxxx-xxxxx-xxxx/providers/Microsoft.Network/virtualNetworks/xxxx-xxxxx-xxxxx-xxxx/subnets/xxxx-xxxxx-xxxxx-xxxx",
	"firstConsecutiveStaticIP": "xxx.xxx.xxx.xxx",
	"distro": "aks-ubuntu-18.04",
            "customVMTags": {
            "Name": "k8s-master",
            "ProjectID": "ICTO-27027",
            "ApplicationID": "AKSE",
            "ApplicationVersion": "v0.50.0",
            "CostCenterID": "8120020",
            "Role": "Shareservice",
            "SupportContact": ""
        }
      },
      "agentPoolProfiles": [
        {
          "name": "pool1",
          "count": 4,
          "vmSize": "Standard_B12ms",
	  "vnetSubnetId": "/subscriptions/xxxx-xxxxx-xxxxx-xxxx/resourceGroups/xxxx-xxxxx-xxxxx-xxxx/providers/Microsoft.Network/virtualNetworks/xxxx-xxxxx-xxxxx-xxxx/subnets/xxxx-xxxxx-xxxxx-xxxx",
	  "distro": "aks-ubuntu-18.04",
          "availabilityProfile": "VirtualMachineScaleSets",
	  "storageProfile": "ManagedDisks",
	  "OSDiskSizeGB": 200,
                         "customVMTags": {
                "Name": "k8s-vmss",
                "ProjectID": "ICTO-27027",
                "ApplicationID": "AKSE",
                "ApplicationVersion": "v0.50.0",
                "CostCenterID": "8120020",
                "Role": "Shareservice",
                "SupportContact": ""
            }
        },
        {
          "name": "pool2",
          "count": 3,
          "vmSize": "Standard_B12ms",
	  "vnetSubnetId": "/subscriptions/xxxx-xxxxx-xxxxx-xxxx/resourceGroups/xxxx-xxxxx-xxxxx-xxxx/providers/Microsoft.Network/virtualNetworks/xxxx-xxxxx-xxxxx-xxxx/subnets/xxxx-xxxxx-xxxxx-xxxx",
	  "distro": "aks-ubuntu-18.04",
          "availabilityProfile": "VirtualMachineScaleSets",
	  "storageProfile": "ManagedDisks",
	  "OSDiskSizeGB": 200,
                         "customVMTags": {
                "Name": "k8s-vmss",
                "ProjectID": "ICTO-27027",
                "ApplicationID": "AKSE",
                "ApplicationVersion": "v0.50.0",
                "CostCenterID": "8120020",
                "Role": "Shareservice",
                "SupportContact": ""
            }
        }
      ],
      "linuxProfile": {
        "adminUsername": "vmadmin",
        "ssh": {
          "publicKeys": [
            {
              "keyData": "ssh-rsa "
            }
          ]
        }        
      },   
      "aadProfile": {
        "serverAppID": "xxxx-xxxxx-xxxxx-xxxx",
        "clientAppID": "xxxx-xxxxx-xxxxx-xxxx",
        "tenantID": "xxxx-xxxxx-xxxxx-xxxx",
        "adminGroupID": "xxxx-xxxxx-xxxxx-xxxx"
    },      
      "certificateProfile": {}
  }
}

生成Azure Resource Manager template

aks-engine  generate akse.json

_output目录下会生成相关文件，建议备份这个目录下的文件，后面集群维护会使用相关配置文件或者证书。

部署集群

aks-engine deploy --resource-group "xxxx-xxxx-xxxx-xxxx \
  --azure-env "AzureChinaCloud" \
  --location "${region}" \
  --subscription-id "${subscription_id}" \
  --client-id "${client_id}" \
  --client-secret "${client_secret}" \
  --api-model "_output/xxxx-xxxx-xxxx-xxxx/apimodel.json" \
  -o "deploy" \
  --debug -f

验证

登录跳本机，安装kubectl

curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.17.11/bin/linux/amd64/kubectl

确认安装，第一次登录集群需要通过aad认证

sudo kubectl get nodes
To sign in, use a web browser to open the page https://microsoft.com/deviceloginchina and enter the code CRXP2BB5U to authenticate.
NAME                                STATUS   ROLES    AGE    VERSION
k8s-master-81692357-0               Ready    master   7m2s   v1.17.11
k8s-master-81692357-1               Ready    master   7m2s   v1.17.11
k8s-master-81692357-2               Ready    master   6m7s   v1.17.11
k8s-prdconapl-81692357-vmss000000   Ready    agent    7m2s   v1.17.11
k8s-prdconapl-81692357-vmss000001   Ready    agent    7m2s   v1.17.11
k8s-prdconapl-81692357-vmss000002   Ready    agent    7m2s   v1.17.11
k8s-prdconeny-81692357-vmss000000   Ready    agent    7m2s   v1.17.11