Apache Airflow2.2.4

 下载

curl -LfO 'https://airflow.apache.org/docs/apache-airflow/2.2.4/docker-compose.yaml'

 

环境

mkdir -p ./dags ./logs ./plugins
echo -e "AIRFLOW_UID=$(id -u)" > .env

 

启动

docker-compose -f docker-compose.yaml up

[root@VM-0-13-centos airflow]# docker-compose -f docker-compose.yaml up
Creating network "airflow_default" with the default driver
Creating volume "airflow_postgres-db-volume" with default driver
Pulling postgres (postgres:13)...
13: Pulling from library/postgres
a603fa5e3b41: Pull complete
02d7a77348fd: Pull complete
Digest: sha256:3c6a77caf1ef2ae91ef1a2cdc2ae219e65e9ea274fbfa0d44af3ec0fccef0d8d
Status: Downloaded newer image for postgres:13
Pulling redis (redis:latest)...
latest: Pulling from library/redis
a603fa5e3b41: Already exists
828da1afb5be: Pull complete
Digest: sha256:1e3207c292225b6dd21cb74d59255748a50e8f739dd983040df38fa913927cf1
Status: Downloaded newer image for redis:latest
Pulling airflow-init (apache/airflow:2.2.4)...
2.2.4: Pulling from apache/airflow
6552179c3509: Pull complete
c4887dad22fd: Pull complete
Digest: sha256:72a2cdcdabbc622c30940f1a9f262d047fdbbe96d5d7a4e324b8c7ec5ef56171
Status: Downloaded newer image for apache/airflow:2.2.4
Creating airflow_redis_1    ... done
Creating airflow_postgres_1 ... done
Creating airflow_airflow-init_1 ... done
Creating airflow_airflow-worker_1    ... done
Creating airflow_flower_1            ... done
Creating airflow_airflow-scheduler_1 ... done
Creating airflow_airflow-triggerer_1 ... done
Creating airflow_airflow-webserver_1 ... done
Attaching to airflow_redis_1, airflow_postgres_1, airflow_airflow-init_1, airflow_airflow-worker_1, airflow_flower_1, airflow_airflow-scheduler_1, airflow_airflow-triggerer_1, airflow_airflow-webserver_1
airflow-scheduler_1  | The container is run as root user. For security, consider using a regular user account.
airflow-init_1       | The container is run as root user. For security, consider using a regular user account.
airflow-init_1       | 
airflow-init_1       | [2022-11-22 14:50:04,935] {cli_action_loggers.py:105} WARNING - Failed to log action with (psycopg2.errors.UndefinedTable) relation "log" does not exist
airflow-init_1       | LINE 1: INSERT INTO log (dttm, dag_id, task_id, event, execution_dat...
airflow-init_1       |                     ^
airflow-init_1       | 
airflow-init_1       | [SQL: INSERT INTO log (dttm, dag_id, task_id, event, execution_date, owner, extra) VALUES (%(dttm)s, %(dag_id)s, %(task_id)s, %(event)s, %(execution_date)s, %(owner)s, %(extra)s) RETURNING log.id]
airflow-init_1       | [parameters: {'dttm': datetime.datetime(2022, 11, 22, 14, 50, 4, 923049, tzinfo=Timezone('UTC')), 'dag_id': None, 'task_id': None, 'event': 'cli_upgradedb', 'execution_date': None, 'owner': 'root', 'extra': '{"host_name": "4cf9127445e0", "full_command": "[\'/home/airflow/.local/bin/airflow\', \'db\', \'upgrade\']"}'}]
airflow-init_1       | (Background on this error at: http://sqlalche.me/e/13/f405)
airflow-init_1       | DB: postgresql+psycopg2://airflow:***@postgres/airflow
airflow-init_1       | [2022-11-22 14:50:05,323] {db.py:919} INFO - Creating tables
airflow-init_1       | INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
airflow-init_1       | INFO  [alembic.runtime.migration] Will assume transactional DDL.
airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade  -> e3a246e0dc1, current schema
airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade e3a246e0dc1 -> 1507a7289a2f, create is_encrypted
airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 1507a7289a2f -> 13eb55f81627, maintain history for compatibility with earlier migrations
airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 13eb55f81627 -> 338e90f54d61, More logging into task_instance
airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 849da589634d -> 2c6edca13270, Resource based permissions.
airflow-init_1       | [2022-11-22 14:50:11,631] {manager.py:245} INFO - Inserted Role: Admin
airflow-init_1       | [2022-11-22 14:50:11,635] {manager.py:245} INFO - Inserted Role: Public
airflow-init_1       | [2022-11-22 14:50:11,637] {manager.py:779} WARNING - No user yet created, use flask fab command to do it.
airflow-init_1       | [2022-11-22 14:50:12,893] {manager.py:496} INFO - Created Permission View: can delete on Connections
airflow-init_1       | [2022-11-22 14:50:12,902] {manager.py:496} INFO - Created Permission View: can read on Connections
airflow-init_1       | [2022-11-22 14:50:13,694] {manager.py:245} INFO - Inserted Role: Viewer
airflow-init_1       | [2022-11-22 14:50:13,702] {manager.py:558} INFO - Added Permission can read on Audit Logs to role Viewer
airflow-init_1       | [2022-11-22 14:50:22,884] {manager.py:496} INFO - Created Permission View: can create on XComs
airflow-init_1       | [2022-11-22 14:50:22,889] {manager.py:558} INFO - Added Permission can create on XComs to role Admin
airflow-init_1       | [2