Business
Jobs
  • About Us
  • Solutions
    • Job Postings
      Post your job and receive qualified candidates in 48h.
    • Candidate Assessments
      500+ technical and psychological tests, plus anti-fraud.
    • Headhunting
      Tailor-made executive search from start to finish.
    • Payroll + EOR
      Payroll dispersal and EOR across 15+ LATAM countries.
  • Pricing
  • Jobs

0

213
Views
Airflow depends_on_past explanation

According to the official Airflow docs, The task instances directly upstream from the task need to be in a success state. Also, if you have set depends_on_past=True, the previous task instance needs to have succeeded (except if it is the first run for that task).

As all know, the task is kind of 'instantiated & parameteriazed' operator.

Now this is what confuse me. For example:

DAG: {op_1} -> {op_2} -> {op_3}

{op_2} is a simple PythonOperator that takes 1 parameter from {op_1} and do stuff;

To my understanding, op_2(param_1) & op_2(param_2) are considered as 2 different tasks.

Given depends_on_past is set to True, then:

  1. If op_2(param_1) is still running; can op_2(param_2) be run?
  2. If op_2(param_1) fails in the previous run; can op_2(param_1) be run in the current run?
over 4 years ago · Santiago Trujillo
1 answers
Answer question

0

From the official docs for trigger rules:

depends_on_past (boolean) when set to True, keeps a task from getting triggered if the previous schedule for the task hasn’t succeeded.

So unless a previous run of your DAG has failed, the depends_on_past should not be a factor, it will not affect the current run at all if the previous run executed the tasks successfully.

over 4 years ago · Santiago Trujillo Report
Answer question
Find remote jobs

Discover the new way to find a job!

Top jobs
Top job categories
Business
Post vacancy Pricing Sales
Legal
Terms and conditions Privacy policy
© 2026 PeakU Inc. All Rights Reserved.
Andres GPT
Show me some job opportunities
There's an error!