Apache Airflow 安装

Apache Airflow 简介

Apache Airflow is an open-source platform for developing, scheduling, and monitoring batch-oriented workflows. Airflow’s extensible Python framework enables you to build workflows connecting with virtually any technology. A web interface helps manage the state of your workflows. Airflow is deployable in many ways, varying from a single process on your laptop to a distributed setup to support even the biggest workflows.

What is Airflow? — Airflow Documentation

GitHub - apache/airflow: Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

Apache Airflow安装

pip install 'apache-airflow==2.6.2' \
 --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.6.2/constraints-3.8.txt"

报错：

 pip install 'apache-airflow==2.6.2' --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.6.2/constraints-3.8.txt" 
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x10e9c7450>: Failed to establish a new connection: [Errno 61] Connection refused')': /apache/airflow/constraints-2.6.2/constraints-3.8.txt
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x10ea98810>: Failed to establish a new connection: [Errno 61] Connection refused')': /apache/airflow/constraints-2.6.2/constraints-3.8.txt
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x10ea98e90>: Failed to establish a new connection: [Errno 61] Connection refused')': /apache/airflow/constraints-2.6.2/constraints-3.8.txt
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x10ea99650>: Failed to establish a new connection: [Errno 61] Connection refused')': /apache/airflow/constraints-2.6.2/constraints-3.8.txt
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x10ea99ed0>: Failed to establish a new connection: [Errno 61] Connection refused')': /apache/airflow/constraints-2.6.2/constraints-3.8.txt
ERROR: Could not install packages due to an OSError: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /apache/airflow/constraints-2.6.2/constraints-3.8.txt (Caused by NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x10ea9a790>: Failed to establish a new connection: [Errno 61] Connection refused'))

Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1002)'))) - skipping

 

解决方案：

pip install 'apache-airflow==2.6.2' \
 --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.6.2/constraints-3.8.txt" --trusted-host pypi.org

把pypi.org加入可信站点即可。