Airflow 2正式出来了

Airflow 2正式出来了

Airflow 2.0.0, 2020-12-17
https://github.com/apache/airflow/releases/tag/2.0.0

1.新的写dag脚本方式。（使用python注释）

@task
def extract():
return {“1001”: 301.27, “1002”: 433.21, “1003”: 502.22}

2.Fully specified REST API (AIP-32)

更全的 REST API 接口。提供外部调用。

3.Massive Scheduler performance improvements

As part of AIP-15 (Scheduler HA+performance) and other work Kamil did, we significantly improved the performance of the Airflow Scheduler. It now starts tasks much, MUCH quicker.
更快的启动任务

4.Scheduler is now HA compatible (AIP-15)

调度高可用。（主备)

5.Task Groups (AIP-34)

可以在任务中建子任务(一个任务中包含多个子任务)，可以在web UI上钻取操作。

6.Refreshed UI

自动刷新介面任务状态。不用手工老点介面刷新了。

7.Smart Sensors for reduced load from sensors (AIP-17)

更智能化的检查器功能。(例如检查其它dag中的任务)，在大量使用检查器时，减小资源使用。

8.Simplified KubernetesExecutor

减化 KubernetesExecutor 执行器。

9.Airflow core and providers: Splitting Airflow into 60+ packages

将airflow 安装包分成多个安装包，可单独下载

10.Configuration

Configuration in the form of the airflow.cfg file has been rationalized further in distinct sections, specifically around “core”. Additionally, a significant amount of configuration options have been deprecated or moved to individual component-specific configuration files, such as the pod-template-file for Kubernetes execution-related configuration.
将配置文件分成多个独立的模块文件。