close

How to prevent airflow from backfilling dag runs?

Hello Guys, How are you all? Hope You all Are Fine. Today We Are Going To learn about How to prevent airflow from backfilling dag runs in Python. So Here I am Explain to you all the possible Methods here.

Without wasting your time, Let’s start This Article.

Table of Contents

How to prevent airflow from backfilling dag runs?

  1. How to prevent airflow from backfilling dag runs?

    Upgrade to airflow version 1.8 and use catchup_by_default=False in the airflow.cfg or apply catchup=False to each of your dags.

  2. prevent airflow from backfilling dag runs

    Upgrade to airflow version 1.8 and use catchup_by_default=False in the airflow.cfg or apply catchup=False to each of your dags.

Method 1

Upgrade to airflow version 1.8 and use catchup_by_default=False in the airflow.cfg or apply catchup=False to each of your dags.

Method 2

Setting catchup=False in your dag declaration will provide this exact functionality.

I don’t have the “reputation” to comment, but I wanted to say that catchup=False was designed (by me) for this exact purpose. In addition, I can verify that in 1.10.1 it is working when set explicitly in the instantiation. However I do not see it working when placed in the default args. I’ve been away from Airflow for 18 months though, so it will be a bit before I can take a look at why the default args isn’t working for catchup.

dag = DAG('example_dag',
        max_active_runs=3,
        catchup=False,
        schedule_interval=timedelta(minutes=5),
        default_args=default_args)

Summery

It’s all About this issue. Hope all Methods helped you a lot. Comment below Your thoughts and your queries. Also, Comment below which Method worked for you? Thank You.

Also, Read