目录
1 什么是airflow?
- It’s a data orchestration and scheduling platform
- it’s a tool to manage your data-flows and data operations
- allow running your task in a distributed manner
- manage your task dependency within the dag.
个人理解为一个任务规划平台,利用Dags( directed acyclic graph)规划task的运行计划以及运行方式. 详细的内容参照123,这里不再赘述,拷贝的再全部都不如先把它跑起来,慢慢体会它的优点和缺点。
2 什么是kubernetes?
1 it is an open source platform for managing containerized workloads and services and it operates at the container level rather than at the hardware level
2 it provides you with a framework to run distributed systems resiliently.
3 It takes care of scaling and failover for your application, provides deployment patterns
4 service discovery and load balancing
5 storage orchestration Kubernetes allows you to automatically mount a storage system of your choice, such as local storages, public cloud providers, and more.
6 Automated rollouts and rollbacks
7 Self-healing Kubernetes restarts containers that fail, replaces containers, kills containers that don’t respond to your user-defined health check, and doesn’t advertise them to clients until they are ready to serve.
9 Secret and configuration management Kubernetes lets you store and manage sensitive information, such as passwords, OAuth tokens, and SSH keys.
10 comprises a set of independent, composable control processes that continuously drive the current state towards the provided
本文介绍了在Kubernetes上部署Airflow的详细步骤,包括使用Kind创建K8s集群,通过Kubectl管理集群,安装Airflow以及使用Helm简化部署流程。重点讲述了如何使用KubernetesOperator和KubernetesExecutor来调度任务。
最低0.47元/天 解锁文章

946

被折叠的 条评论
为什么被折叠?



