Most data platform teams answer the ETL versus ELT question once, at the platform level, and apply that answer to every pipeline.
The reasoning is understandable.
One pattern means fewer tools, fewer skills to maintain, fewer failure modes, and simpler runbooks. Standardization helps platform teams stay sane.
Then the portfolio grows.
A pipeline loads terabytes of raw data into the warehouse, performs an expensive transformation, and produces a small aggregate. The organization pays warehouse compute to process data that could have been reduced earlier.
Another pipeline transforms too aggressively before loading. Six months later, an analyst needs detail that was discarded during ingestion. Reconstructing it requires a source-system project nobody planned for.
The pattern choice is not a permanent platform standard.
It is a per-pipeline decision with cost, latency, flexibility, and compliance consequences.
What ETL and ELT Actually Trade
ETL means extract, transform, then load.
The data is transformed before it reaches the analytical destination.
ELT means extract, load, then transform.
The raw or lightly processed data lands first, and transformations happen inside the warehouse or lakehouse.
Neither pattern is universally better.
ETL is useful when you need to reduce data volume before loading, apply controls before broader access, or run stable and computationally heavy transformations outside the warehouse.
ELT is useful when you want to preserve raw data, iterate quickly on transformation logic, and let analysts modify SQL without requiring an engineering deployment.
The mistake is treating either pattern as an ideology.
Why the Wrong Choice Appears in the Warehouse Bill
ELT deliberately uses warehouse compute for transformation.
That is often a good trade. You preserve raw data, simplify ingestion, and keep business logic close to the analysts who change it.
The trade becomes less attractive when:
The transformation runs on every load
The input is much larger than the output
The transformation is computationally heavy
The output discards most of the source data
The warehouse is already capacity-constrained
Loading two terabytes to produce a four-gigabyte aggregate may still be the right choice if the raw data has analytical value.
But if the raw data is never queried, must be reduced before access, and is expensive to transform repeatedly, ETL deserves serious consideration.
Do not optimize based on the size of the final table alone. Compare:
Input volume
Output volume
Transformation frequency
Compute cost
Retention requirements
Future analytical value of the raw data
Why Latency Changes the Answer
ETL adds a processing stage before the data reaches its destination.
That can increase freshness latency.
For a dashboard that genuinely requires data every five minutes, an additional transformation stage may be unacceptable. ELT, micro-batching, or streaming may be a better fit.
But many freshness requirements are inherited rather than measured.
A team says a dashboard needs five-minute freshness because that was written in an old requirements document. The dashboard is reviewed once each morning.
That is not a five-minute business requirement. It is an unchallenged assumption.
Before choosing a lower-latency pattern, ask how quickly someone must act on the data. Freshness is valuable when it changes a decision, not merely because the pipeline can be made faster.
Why Compliance Changes the Answer
Compliance does not automatically mean every sensitive pipeline must use ETL.
The real question is:
Can raw sensitive data land in the destination under an access model that prevents unauthorized users from seeing it?
If the answer is no, sensitive fields must be masked, tokenized, filtered, or otherwise controlled before they reach the broadly accessible analytical layer.
That may require ETL.
In other architectures, raw data can land in a tightly restricted zone while a governed transformation produces the analyst-facing tables.
The requirement is not “always use ETL.”
The requirement is that sensitive data does not become broadly accessible before the required controls are applied.
Document where the control happens, who can access the raw layer, and how the transformed layer is verified.
The Four Questions to Ask for Every Pipeline
1. What is the input-to-output ratio?
If a pipeline consistently reduces a large source into a much smaller analytical output, evaluate ETL.
A high reduction ratio does not automatically settle the decision. Raw data may still be valuable for future analysis, audits, or reprocessing.
But repeatedly paying warehouse compute to discard most of the input is a clear signal to investigate.
2. Does the pipeline contain sensitive data?
Identify whether the source includes:
Personal information
Financial information
Health information
Credentials or secrets
Customer-level behavioral data
Regulated operational data
Then define the first point at which unauthorized access is technically prevented.
If sensitive data must be controlled before it reaches the general warehouse, transform before that boundary.
3. How stable is the transformation logic?
Frequently changing business logic often belongs in ELT.
Analysts can iterate in SQL, compare versions, and update transformations without rebuilding the ingestion system.
Stable, computationally heavy transformations are stronger ETL candidates, especially when they reduce data substantially before loading.
Do not confuse “stable” with “important.” A metric can be business-critical and still change frequently.
4. What freshness requirement is real?
Classify the pipeline based on the action it supports:
Daily reporting
Hourly operational review
Near-real-time monitoring
Automated decisioning
User-facing product behavior
Then measure the acceptable delay.
A pipeline that supports daily reporting does not need a five-minute architecture. A pipeline that triggers fraud controls may.
A Practical Decision Rule
Use ETL when most of the following are true:
The transformation is computationally heavy
The output is dramatically smaller than the input
The logic is stable
Sensitive fields must be controlled before broad access
Extra processing latency is acceptable
Use ELT when most of the following are true:
Raw data has ongoing analytical value
Transformation logic changes frequently
Analysts need direct SQL control
The pipeline requires very low latency
Warehouse capacity can support the transformation workload
When the signals conflict, resolve them in this order:
Security and compliance requirements
Real freshness requirement
Input-to-output economics
Transformation volatility
Platform simplicity
This keeps cost from overriding a control requirement and prevents inherited latency assumptions from dictating every design.
What Per-Pipeline Decisions Return
Teams that evaluate pipelines individually usually end up with a portfolio rather than a religion.
High-reduction and control-sensitive pipelines use ETL where it creates a clear economic or security advantage.
Stable-volume pipelines with frequently changing business logic use ELT where analyst flexibility is worth the warehouse cost.
The platform still has standards.
The standard is not “everything uses ETL” or “everything uses ELT.”
The standard is the decision process.
Track the choice for each pipeline:
Selected pattern
Reason for selection
Expected cost profile
Freshness target
Data sensitivity
Review trigger
Revisit the choice when the pipeline’s volume, consumers, sensitivity, or latency requirement changes.
Supporting two patterns creates real operational overhead.
It is still cheaper than forcing every pipeline into a pattern that fits half the portfolio.
A platform default is useful.
A default nobody is allowed to revisit becomes technical debt with a governance committee.
That’s it for today!
Did you enjoy this newsletter issue?
Share with your friends, colleagues, and your favorite social media platform.
Until next week — Amrut
Get in touch
You can find me on LinkedIn or X.
If you would like to request a topic to read, please feel free to contact me directly via LinkedIn or X.
Good one ! You captured neatly on fundamentals- it helps someone wants to learn basics of data engineering. I enjoyed the article.