When I trying to import a script located in a folder outside dags I got a
ModuleNotFoundError: No module named 'scripts.send_email'
or
ImportError: cannot import name 'send_email' from 'scripts' (unknownlocation)
I've tried the following ways:
from airflow.scripts.send_email import send_email #ModuleNotFoundErrorfrom scripts.send_email import send_email #ModuleNotFoundErrorfrom scripts import send_email # ImportErrorMy Airflow Project has the following structure
airflow├── config├── dags├── __init__.py├── coinbase_fetch.py├── coinbase_orchestration.py└── load_into_minio.py ├── logs├── plugins├── scripts├── __init__.py└── send_email.py├── __init__.py├── docker-compose.py├── Dockerfile└── requirements.txtThe script I try to load in scripts/send_email.py is only an example which return a string. inside send email you'll find:
def send_email(): print("SEND EMAIL") I've added to the dockerfile the PYTHONPATH as I read here Modules Management. Therefore my Dockerfile is as you can see here:
FROM apache/airflow:2.9.0ENV PYTHONPATH "${PYTHONPATH}:/usr/local/airflow/scripts"ENV AIRFLOW__CORE__LOAD_EXAMPLES=TrueENV AIRFLOW__DATABASE__SQL_ALCHEMY_CONN=my_conn_stringCOPY requirements.txt /RUN pip install --no-cache-dir "apache-airflow==${AIRFLOW_VERSION}" -r /requirements.txtI've also added the scripts file in the docker-compose volumes
volumes: - ${AIRFLOW_PROJ_DIR:-.}/config:/opt/airflow/config - ${AIRFLOW_PROJ_DIR:-.}/dags:/opt/airflow/dags - ${AIRFLOW_PROJ_DIR:-.}/logs:/opt/airflow/logs - ${AIRFLOW_PROJ_DIR:-.}/plugins:/opt/airflow/plugins - ${AIRFLOW_PROJ_DIR:-.}/scripts:/opt/airflow/scripts #Added However I couldn't import the module inside the script folder.
I've also tried using sys.path.insert or sys.path.append as was mentioned in the links below without success:
The full project is in this github in the folder airflow: WIP Airflow