kubernetes helm

术语

• Chart：一个helm程序包
• Repository：Charts仓库，https/http服务器
• Release：特定的Chart部署于目标集群上的一个实例
• Config: values.yaml

Chart -> Config -> Release

程序架构

helm：客户端，管理本地的Chart仓库，管理Chart, 与Tiller服务器交互，发送Chart，实例安装、查询、卸载等操作

Tiller：服务端，接收helm发来的Charts与Config，合并生成relase；

RBAC配置文件示例：
https://github.com/helm/helm/blob/master/docs/rbac.md

官方可用的Chart列表：
https://hub.kubeapps.com/

helm

https://helm.sh/

https://github.com/helm/helm

https://github.com/helm/helm/releases

wget https://storage.googleapis.com/kubernetes-helm/helm-v2.12.3-linux-amd64.tar.gz

tar xf helm-v2.12.3-linux-amd64.tar.gz
cd linux-amd64/
mv helm tiller /usr/bin/

vim /etc/profile
source <(helm completion bash)

安装tiller

yum install socat

创建rbac角色

helm-rbac.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: tiller
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: tiller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - kind: ServiceAccount
    name: tiller
    namespace: kube-system

添加国内源(备用)

helm repo add stable https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts

下载镜像

docker pull registry.cn-beijing.aliyuncs.com/minminmsn/tiller:v2.12.3
docker tag registry.cn-beijing.aliyuncs.com/minminmsn/tiller:v2.12.3 gcr.io/kubernetes-helm/tiller:v2.12.3

helm init

helm version
helm list

报错:

[root@k8s-master1 helm]# helm list
Error: configmaps is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "configmaps" in API group "" in the namespace "kube-system"

重启tiller即可

helm init --upgrade --service-account tiller

仓库操作

更新仓库

helm repo update

查看仓库

helm repo list

添加仓库

helm repo add incubator http://storage.googleapis.com/kubernetes-charts-incubator

删除仓库

helm repo remove incubator

helm常用命令

release管理

install
delete
upgrade/rollback
list
history：release的历史信息；
status：获取release状态信息；

helm install \
  --set image=redis \
  --set tag=5.0.5-alpine \
    stable/redis-ha


helm install --name redis1 stable/redis

helm install --name redis1 -f values.yaml stable/redis

helm install --values=myvals.yaml wordpress

helm search redis

helm inspect stable/redis

查看release状态

helm status redis1

chart管理

https://v2.helm.sh/docs/developing_charts/

create
fetch    download a chart from a repository and (optionally) unpack it in local directory
get
inspect
package
verify

helm fetch stable/redis

/root/.helm/cache/archive
tree redis
redis
├── Chart.yaml
├── ci
│   ├── default-values.yaml
│   ├── dev-values.yaml
│   ├── extra-flags-values.yaml
│   ├── insecure-sentinel-values.yaml
│   ├── production-sentinel-values.yaml
│   ├── production-values.yaml
│   ├── redisgraph-module-values.yaml
│   └── redis-lib-values.yaml
├── README.md
├── templates
│   ├── configmap.yaml
│   ├── headless-svc.yaml
│   ├── health-configmap.yaml
│   ├── _helpers.tpl
│   ├── metrics-prometheus.yaml
│   ├── metrics-svc.yaml
│   ├── networkpolicy.yaml
│   ├── NOTES.txt
│   ├── psp.yaml
│   ├── redis-master-statefulset.yaml
│   ├── redis-master-svc.yaml
│   ├── redis-rolebinding.yaml
│   ├── redis-role.yaml
│   ├── redis-serviceaccount.yaml
│   ├── redis-slave-statefulset.yaml
│   ├── redis-slave-svc.yaml
│   ├── redis-with-sentinel-svc.yaml
│   └── secret.yaml
├── values-production.yaml
├── values.schema.json
└── values.yaml

chart结构

wordpress/
  Chart.yaml          # A YAML file containing information about the chart
  LICENSE             # OPTIONAL: A plain text file containing the license for the chart
  README.md           # OPTIONAL: A human-readable README file
  requirements.yaml   # OPTIONAL: A YAML file listing dependencies for the chart
  values.yaml         # The default configuration values for this chart
  charts/             # A directory containing any charts upon which this chart depends.
  templates/          # A directory of templates that, when combined with values,
                      # will generate valid Kubernetes manifest files.
  templates/NOTES.txt # OPTIONAL: A plain text file containing short usage notes

Chart.yaml

apiVersion: The chart API version, always "v1" (required)
name: The name of the chart (required)
version: A SemVer 2 version (required)
kubeVersion: A SemVer range of compatible Kubernetes versions (optional)
description: A single-sentence description of this project (optional)
keywords:
  - A list of keywords about this project (optional)
home: The URL of this project's home page (optional)
sources:
  - A list of URLs to source code for this project (optional)
maintainers: # (optional)
  - name: The maintainer's name (required for each maintainer)
    email: The maintainer's email (optional for each maintainer)
    url: A URL for the maintainer (optional for each maintainer)
engine: gotpl # The name of the template engine (optional, defaults to gotpl)
icon: A URL to an SVG or PNG image to be used as an icon (optional).
appVersion: The version of the app that this contains (optional). This needn't be SemVer.
deprecated: Whether this chart is deprecated (optional, boolean)
tillerVersion: The version of Tiller that this chart requires. This should be expressed as a SemVer range: ">2.0.0" (optional)

Dependencies with requirements.yaml

dependencies:
  - name: apache
    version: 1.2.3
    repository: http://example.com/charts
  - name: mysql
    version: 3.2.1
    repository: http://another.example.com/charts

helm dep up foochart

charts/
  apache-1.2.3.tgz
  mysql-3.2.1.tgz

templates/

https://golang.org/pkg/text/template/

apiVersion: v1
kind: ReplicationController
metadata:
  name: deis-database
  namespace: deis
  labels:
    app.kubernetes.io/managed-by: deis
spec:
  replicas: 1
  selector:
    app.kubernetes.io/name: deis-database
  template:
    metadata:
      labels:
        app.kubernetes.io/name: deis-database
    spec:
      serviceAccount: deis-database
      containers:
        - name: deis-database
          image: {{.Values.imageRegistry}}/postgres:{{.Values.dockerTag}}
          imagePullPolicy: {{.Values.pullPolicy}}
          ports:
            - containerPort: 5432
          env:
            - name: DATABASE_STORAGE
              value: {{default "minio" .Values.storage}}

通常在 values.yaml 文件中定义

values.yaml

imageRegistry: "quay.io/deis"
dockerTag: "latest"
pullPolicy: "Always"
storage: "s3"

helm install --values=myvals.yaml wordpress

自定义chart

创建chart

helm create myapp

tree myapp/
myapp/
├── charts
├── Chart.yaml
├── templates
│   ├── deployment.yaml
│   ├── _helpers.tpl
│   ├── ingress.yaml
│   ├── NOTES.txt
│   ├── service.yaml
│   └── tests
│       └── test-connection.yaml
└── values.yaml

myapp/Chart.yaml

apiVersion: v1
appVersion: "1.0"
description: A Helm chart for Kubernetes myapp chart
name: myapp
version: 0.0.1
maintainer:
- name: wuxingge
  email: wuxingge@wuxingge.com
  url: http://www.wuxingge.com/

语法检查

helm lint myapp

打包chart

helm package myapp/

helm仓库服务器

[root@k8s-master1 ~]# helm serve
Regenerating index. This may take a moment.
Now serving you on 127.0.0.1:8879

搜索chart

helm search myapp

部署本地chart

helm install --name myapp3 local/myapp

删除

helm delete --purge myapp3

#查看chart
root@localhost:/path/to/dev-kibana# ls
Chart.yaml  Makefile  README.md  examples  templates  values.yaml

#helm部署
root@localhost:/path/to/dev-kibana# helm --kubeconfig /path/to/kubeconfig.yaml install  dev-kibana -n kibana -f values.yaml .

ELFK

E: elasticsearch
L: logstash
F: Filebeat, Fluentd
K: Kibana

helm fetch incubator/elasticsearch

docker pull registry.cn-hangzhou.aliyuncs.com/wuxingge/elasticsearch-oss:6.4.2
docker tag registry.cn-hangzhou.aliyuncs.com/wuxingge/elasticsearch-oss:6.4.2 docker.elastic.co/elasticsearch/elasticsearch-oss:6.4.2

helm install --name els1 --namespace=efk -f values.yaml incubator/elasticsearch

helm fetch stable/fluentd-elasticsearch

helm fetch stable/kibana