New to Apache Airflow and curious about how code and data are expected to be used across worker nodes in a multinode airflow setup.
When considering if ETL logic should be in the dags or in separate files called by operators in the dag, from here (https://www.astronomer.io/guides/dag-best-practices/) I see:
Try to treat the DAG file like a config file and leave all the heavy lifting for the hook and operator.
From this, my question is: 1) what is best way to sync the operator code across workers in a multinode airflow setup? And 2) given a multinode setup, are we expected to mount shared drives between all workers in order to store intermediate data (or does airflow have some other internal way to pass data between workers to complete dag tasks)?