Technical interview guide

Data Pipeline Orchestration

Scheduling and sequencing interdependent pipeline steps as a DAG, with retries and backfills.

Read: 29 min
Practice MCQs: 25
Interview QA: 25
Edition: v4
Editorial status: Reviewed
Relevant for: Data ArchitectData EngineerForward Deployed Engineer (FDE)

Scope: Apache Airflow 3.3.x, Prefect 3, Kubernetes 1.36, and OpenLineage 1.53 documentation current 2026-08-31.

Interview QA

Treat each question like a live interview question: answer out loud first (structure, assumptions, tradeoffs), then open the model answer to spot gaps and rehearse a tighter follow-up.

Curated: 2026-10-05 · Written: 2026-09-11 · Reviewed: 2026-10-03

QA-1

Design orchestration for a daily source-to-warehouse data product.

QA-2

Make an orchestration task safe for retries and task clearing.

QA-3

Design a retry, timeout, and failure-classification policy.

QA-4

Run a production backfill without harming live freshness or correctness.

QA-5

Choose schedule-, sensor-, or event-triggered orchestration.

QA-6

Design a safe sensor for an external dataset or API condition.

QA-7

Use dynamic task mapping for partitioned work safely.

QA-8

Design concurrency, pools, priorities, and fairness for shared orchestration.

QA-9

Deploy an orchestration workflow change safely.

QA-10

Define orchestration observability and service objectives.

QA-11

Secure an orchestration control plane and its task execution.

QA-12

Design multi-tenant orchestration for many product teams.

QA-13

Orchestrate a task that calls a non-transactional external API.

QA-14

Design compensation across a multi-step business workflow.

QA-15

Recover orchestration after loss or corruption of its metadata database.

QA-16

Design highly available scheduling and execution for critical workflows.

QA-17

Build a test strategy for workflow definitions and orchestration behavior.

QA-18

Coordinate dependencies between independently owned workflows.

QA-19

Add a human approval step to a production workflow.

QA-20

Control orchestration cost without hiding failure or weakening recovery.

QA-21

Migrate a collection of cron jobs to a workflow orchestrator.

QA-22

Respond to an orchestration incident causing stale or partial data.

QA-23

Manage secrets and connections for orchestrated tasks.

QA-24

Capacity-test an orchestration platform and its critical dependencies.

QA-25