Sample questions
Data Pipeline OrchestrationDifficulty 1
In pipeline orchestration, what does a DAG (directed acyclic graph) primarily represent?
- aA task's retry loop
- bDependency order, no cycles✓
- cThe servers a pipeline runs on
- dThe output data's storage format
Explanation:A DAG models tasks as nodes and dependencies as directed edges. 'Acyclic' means no task can depend on itself through a chain, which guarantees a valid execution order can always be computed.
Data Pipeline OrchestrationDifficulty 1
Why must a pipeline dependency graph be acyclic?
- aAcyclic graphs use less disk space
- bCyclic graphs cannot store two tasks
- cA cycle blocks any valid run order✓
- dSchedulers need alphabetical task names
Explanation:If task A depends on B and B depends on A, neither can ever run first — there is no starting point. Acyclicity guarantees at least one valid topological ordering exists.
Data Pipeline OrchestrationDifficulty 2
What is the main difference between cron-style scheduling and event-based triggering for a pipeline?
- aCron is time-driven; events react✓
- bCron runs yearly, event-based every second
- cEvent-based needs no monitoring
- dCron cannot combine with retries
Explanation:Cron-style scheduling is time-driven (e.g. every day at 02:00). Event-based triggering is driven by an occurrence, like a file landing or an upstream pipeline finishing, so the run time varies with when the event happens.
Data Pipeline OrchestrationDifficulty 2
A daily report pipeline must run only after a specific upstream file arrives in storage, and the arrival time varies by a few hours each day. Which scheduling approach fits best?
- aCron set to the earliest possible arrival
- bCron set to the latest possible arrival
- cNo scheduling, run it manually
- dAn event-based trigger on file arrival✓
Explanation:Since arrival time is variable, a fixed cron schedule either risks running too early (file missing) or wastes time waiting until the latest possible slot. An event-based trigger reacts exactly when the dependency is satisfied.
Data Pipeline OrchestrationDifficulty 2
What does 'backfill' mean in the context of pipeline orchestration?
- aDeleting old pipeline runs
- bRe-running the pipeline for past dates✓
- cAdding a new column to a table
- dIncreasing parallel workers for future runs
Explanation:Backfill means executing the pipeline logic against historical time windows — for example running the last 30 days again after a bug fix — so that historical output matches what the corrected logic would have produced.
Data Pipeline OrchestrationDifficulty 2
A pipeline that computes daily active user counts had a bug for the last 10 days that undercounted users. The bug is now fixed. What should typically happen next?
- aNothing, only future runs matter
- bDelete the last 10 days permanently
- cBackfill the affected 10 days✓
- dChange the schedule to run twice daily
Explanation:Since historical output is now known to be wrong, a backfill re-runs the corrected logic over the affected date range so historical data becomes consistent with the fix, rather than leaving stale or missing data.