Airflow DAG Retry Backoff & Critical Path Simulator
Configure retries the way you would for a flaky API, then lay out a DAG with a parallel branch and find its critical path โ the thing that decides how long the run takes.
Predict first
Commit to a guess before you touch anything. Being wrong here is how it sticks.
retries = 4, retry_delay = 10 s, exponential backoff. Total wait if all fail?
Experiment
Move the controls and watch the outputs. Goals tick off automatically when you reach them.
Retry backoff
Critical path
ingest โ (validate_schema โฅ profile_stats) โ clean_and_store
The slower branch (profile_stats) sets the pace. Speeding up the other branch saves nothing.
Explain it
Put what you saw into words. This is what the exam will ask you to do.
Why should ingest get more retries than validate, and what happens downstream when validate fails?
Code it
Write real code against real data. Python and SQL run right here in your browser and hidden tests check your work; the AI reviews it like an examiner.
Retry budgets and the critical path
Two calculations you'll do in every Airflow design question:
backoff_schedule(delay, retries, exponential=True)โ list of waits in seconds before each retry ([20, 40, 80]for delay 20, 3 retries;[20, 20, 20]without exponential backoff).critical_path(durations, upstream)โ the run time of the whole DAG, i.e. the longest path.durationsmaps task โ seconds;upstreammaps task โ list of tasks it depends on.
Challenge
Fixed variations, a new problem every day, and unlimited AI challenges built from your current settings.
Airflow Exponential Backoff Total Wait
An ingest task is configured with retries=3, initial retry_delay=20 seconds, and retry_exponential_backoff=True (doubling each retry: 20s, 40s, 80s). If all retries fail, how many total seconds were spent waiting in retry delays?