TCP#136: You picked your secret store by cost. That was the wrong axis.
Secrets Manager, Parameter Store, and KMS-encrypted env compared on the axes that actually decide the right store per secret.
Most teams choose a secret store the same way they choose a compute platform: by preference or cost, not by what the secret actually needs.
One team saw Secrets Manager’s per‑secret pricing, decided it was expensive, and standardized on Parameter Store for everything. Another standardized on Secrets Manager because it sounded “purpose‑built.” A third never decided at all:
Database passwords in SSM parameters
API keys in Secrets Manager
A few credentials in KMS‑encrypted env vars because that was fastest for one service two years ago
Different routes, same destination: secrets live where habit or a cost comparison put them, not where the secret’s requirements put them.
And those requirements differ: a rotating database password is not the same as a static third‑party API key, which is not the same as a config value that’s merely sensitive.
Optimizing for the cheapest monthly line item while ignoring rotation, audit, and incident response is optimizing the wrong variable.
Why the Wrong Store Is a Real Cost
A secret in the wrong store is not a billing issue. It is:
A rotation problem
An audit problem
An incident‑response problem waiting for its moment
Rotation cost:
A database credential in plain Parameter Store does not rotate itself. Someone has to remember. They don’t. A credential that should live 30 days lives 2 years. When it leaks, the blast radius is 2 years of access, not 30 days.
Audit cost:
Compliance frameworks want proof: scheduled rotation, scoped access, logged reads. A secret in the wrong store cannot show that history. The auditor asks for rotation records, and there are none, because that store doesn’t rotate.
Incident cost:
When a credential leaks, the first questions are:
How old is it?
What can it reach?
How fast can we rotate or revoke it?
The store built for rotation answers quickly. The store chosen for price does not, and the incident stretches from hours into days.
The monthly price delta between stores is small.
The cost of a credential you cannot rotate, audit, or quickly revoke is very high, and shows up exactly when you can least afford it.
How Teams End Up With Secrets Scattered
Sprawl is not usually a decision. It’s the absence of one.
The first secret goes “wherever” because one engineer had to ship one service.
The second secret goes somewhere else because another engineer made a different local call.
Neither choice is crazy in isolation. Together, they are the start of chaos.
Then cost anxiety accelerates it:
Secrets Manager’s per‑secret line is visible on the bill
Parameter Store’s usage is fuzzier
A cost review flags Secrets Manager
The team moves secrets to the “cheaper” store without checking whether it meets the secrets’ requirements
The bill goes down. Rotation and audit capability go with it.
This persists because nobody owns the decision. There’s a standard for compute, a standard for networking, and no standard for secrets. So the question “where does this go?” gets answered per engineer, per service, per deadline.
Sprawl is the predictable result of a decision nobody was assigned to make.
The Axes That Actually Decide
The “right” store is not about preference or price. It’s about four properties of the secret itself. Score the secret on these, and the store largely selects itself.
Does it need automatic rotation?
This is the first cut.Must‑rotate (e.g. most DB credentials, many service credentials) → Secrets Manager, with native rotation where possible
Does‑not‑rotate (static third‑party key, certain config values) → does not need to pay for rotation it won’t use
How sensitive is it and what’s the blast radius if it leaks?
High‑blast‑radius (prod DB with customer data) → strongest access controls, logging, and rotation
Lower sensitivity (non‑public but low‑impact config) → can live in a simpler store
How often is it read, and by what?
Read pattern affects both architecture and cost:Read on every request by a high‑traffic service → needs caching and cost awareness
Read once at startup → throughput less relevant
Parameter Store and Secrets Manager behave differently under load; the wrong choice here is either a latency issue or a surprise bill.
What’s its compliance scope?
In‑scope for FedRAMP / HIPAA / ISO? Then you may be forced into:Demonstrable rotation
Scoped IAM
Auditable access history
“We cannot demonstrate rotation” is a finding all by itself.
KMS‑encrypted environment variables sit at the bottom of this matrix on purpose. They are right for a narrow case:
Value needed only at process start
Not sensitive enough to justify a lookup per read
Does not rotate
They are wrong for almost everything else. When secrets land here, it’s because no one applied the matrix at all.
What a Standard Changes
Teams that define where each class of secret belongs stop putting secrets in the wrong place.
Rotating database credentials live where rotation is native, so they actually rotate, and a leak’s blast radius is 30 days, not 2 years.
Static API keys live where they aren’t paying for rotation they don’t need.
High‑sensitivity secrets live where access is tightly scoped and logged, so audits pull reports instead of doing archaeology.
Low‑risk config values live in the simple store, so the expensive store is reserved for secrets that justify it.
When a credential leaks, the response is fast because the store was chosen for that risk class. The team knows its age, reach, and rotation path because the store was designed to answer those questions.
Secrets stop being scattered by accident and start being placed by design. The store follows the secret’s requirements, and the requirements were explicitly examined instead of assumed.
That’s it for today!
Did you enjoy this newsletter issue?
Share with your friends, colleagues, and your favorite social media platform.
Until next week — Amrut
Get in touch
You can find me on LinkedIn or X.
If you would like to request a topic to read, please feel free to contact me directly via LinkedIn or X.