ETL versus ELT is a placement decision, not a religion

ETL extracts data, transforms it in a processing system, then loads the result into a target. ELT extracts and loads source-shaped data first, then transforms it using the destination or an adjacent scalable engine. Modern analytical warehouses commonly favor ELT because storage is relatively inexpensive, compute scales independently, SQL transformation is accessible, and retained source-shaped history supports new models and replay. ETL remains appropriate when sensitive or prohibited data must be removed before landing, the target cannot process the source volume or format economically, strict validation must precede acceptance, network movement must be reduced, or mature specialized processing already exists. Real platforms often use a hybrid: minimal extraction-time validation, classification and protection; durable landing; then modeled transformations.

Separate ingestion truth from analytical meaning

An ingestion layer records what a source produced, including source identity, extraction window or change position, schema/version, event or commit time, arrival time, batch/run identity, checksums and deletion semantics. "Raw" does not mean ungoverned or literally byte-for-byte forever: secrets and prohibited fields may require source-side exclusion, data needs encryption and access control, and formats may be normalized when provenance and meaning remain reconstructable. Curated layers apply deduplication, type handling, identity resolution, business rules, dimensions, facts, aggregates and serving contracts. Consumers should not depend accidentally on unstable landing schemas.

Correctness is harder than transform placement

Pipelines need explicit snapshot, append, upsert and delete semantics. Incremental processing records stable source positions or watermarks and defines lookback, overlap, deduplication, late arrival, ordering and replay. Change data capture can lower latency and source load, but log positions, transaction boundaries, initial snapshots, schema changes, tombstones, slot retention and failover must be operated carefully. Exactly-once business outcomes usually come from idempotent writes, deterministic keys, transactional commits and replay-aware sinks—not a blanket promise from the orchestrator.

Schema and data contracts distinguish additive, breaking and semantic change. Preserve unknown fields and quarantine incompatible records when appropriate, alert owners, and avoid silently coercing corrupt data to null. Backfills use immutable code/config versions, bounded source snapshots, isolated computation, reconciliation and atomic publication so historical correction does not mix partial versions with live results. Slowly changing dimensions and effective-time logic must separate when a fact happened from when the platform learned it.

Operate data as a product

Orchestration coordinates dependencies, retries, concurrency, schedules and backfills; it does not make non-idempotent tasks safe. Every dataset needs ownership, purpose, schema, freshness and quality expectations, lineage, access classification, retention, cost and incident behavior. Quality checks cover volume and freshness plus uniqueness, completeness, validity, referential integrity, distribution drift, reconciliation to sources and business invariants. Publish run and dataset lineage with code and version context, but verify completeness—lineage tools cannot see transformations or copies they do not instrument.

Placement mechanics from warehouse and lakehouse engines

Google Cloud's load-transform-export guidance treats ETL, ELT, and reverse ETL as placement around BigQuery: transform elsewhere then load; load then SQL-transform in place; or export warehouse results back to operational systems. Reverse ETL is a governed export with its own identity, overwrite, and freshness contract—not “ELT run backwards.” Dataform then versions SQL models, tests, documentation, and schedules as code rather than console one-offs. Airflow can schedule those jobs, but a successful DAG run still does not prove the warehouse merge was idempotent.

AWS Glue's serverless ETL still pays for shuffle, worker memory, and file layout. Glue best practices emphasize converting many small objects into fewer columnar Parquet files, partitioning on keys that match filters, and avoiding wide untyped JSON that forces full scans. A Glue job that “succeeds” after coercing corrupt fields to null is a correctness failure, not a format win. Spark Structured Streaming can apply the same transforms incrementally, but watermarks, state stores, and checkpoint directories are part of the contract: deleting a checkpoint to “unstick” a job can reprocess or skip source positions.

PostgreSQL logical decoding is a common CDC extract. A replication slot retains WAL until every consumer acknowledges an LSN. If the warehouse lags, slot retention can fill disk and stall the source; failover without slot following can drop a gap that later ELT treats as a clean snapshot. Iceberg reliability matters at load time: table commits are atomic with serializable isolation and optimistic concurrency, so two loaders rewriting the same partition must retry on conflict rather than leaving mixed manifests. OpenLineage job, run, and dataset events should bind extract position, transform version, and published snapshot; missing instrumentation around Glue or Dataform is a lineage hole, not proof that nothing moved.

Failure modes that survive a green load

Count mismatches of a few percent after ELT often come from timezone conversion at extract versus warehouse, duplicate CDC updates without a merge key, or deletes represented as later inserts. Parquet rewards predicate pushdown only when types and column statistics are honest; a stringly typed timestamp will not prune. When landing “raw” JSON, preserve a checksum or byte-level identity so replay can prove the warehouse computed the same extract. Hybrid pipelines that classify at extract, land source-shaped history, then model in SQL should still quarantine contract failures instead of letting warehouse CAST hide them.

Optimize cost only after correctness and service objectives are explicit. Use columnar formats, compression, partitioning, clustering, incremental models and workload isolation to reduce scanning and movement; monitor small files, skew, spill, shuffle, slot/warehouse use and repeated transformations. Security applies across sources, transport, landing, transformation, orchestration metadata, logs, lower environments and exports. Measure end-to-end freshness, completeness, reconciliation, failed and late records, recovery and replay time, consumer incidents, lineage coverage and cost per useful workload. Choose ETL, ELT or hybrid per flow based on governance, correctness, latency, skills, compute, portability and recovery—not fashion.

Worked example: 10,000 orders, 40 bad SKUs

Source checkout: 10,000 rows, $248,000 revenue. Forty rows have SKU "". Finance closes from warehouse SUM(amount) joined to the product table.

placement	warehouse COUNT(*)	null SKUs	SUM(amount) vs source	close the books
ETL reject-then-load	9,960	0	$12,400 short, quarantine file lists 40	yes, with an exception report
ELT CAST-to-null	10,000	40	join drops 40; dashboard still shows 10,000 rows	no — silent $12,400 hole
hybrid: quarantine + land	9,960 modeled + 40 in bad_sku	0 in mart	mart matches 9,960; bad_sku is owned	yes

A green Glue/Dataform run is not the interview. The interview is whether row count, dollars, and exceptions reconcile to the same 10,000.