[Security Architecture] Secret Management - Going Secret-less Is the Best Defense

About this article

As the seventh installment of the “Security Architecture” category in the series “Architecture Crash Course for the Generative-AI Era,” this article explains secret management.

The mechanisms for storing, distributing, and rotating information like API keys, passwords, certificates, and tokens - things where one person’s slip-up tilts the whole company. AWS Access Keys accidentally committed to GitHub get picked up by bots in seconds, and gigantic Bitcoin-mining instances spin up uninvited - secret leaks lead directly to tens of millions of yen of damage in hours. This article covers selection of Vault/Secrets Manager/Cloud KMS, rotation, least privilege, CI/CD integration, and accidental-commit detection.

Why secret management is needed

Leakage directly becomes unauthorized access

If secrets leak, attackers can operate as legitimate users or systems. WAFs, MFA, and authorization rules all let through opponents holding legitimate keys. Defense mechanisms become powerless and damage spikes.

Accidental GitHub commits happen often

“Accidentally committed .env,” “printed an API key in a log for debugging” - these accidents happen somewhere every day. Once it’s gone public, it’s not “OK to delete from history” - it’s immediate rotation required.

Need to satisfy regulatory requirements

SOC 2 (the security audit that’s effectively standard for US-based SaaS), ISO 27001, PCI DSS - all require safe management of secrets. If audits find plaintext secrets in Git repos or wikis, it’s an instant red card.

Types of secrets

Secrets to defend span widely. Even one leaking can be fatal, so the rule is to manage all types with the same strictness.

Type	Example
API keys	External services like Stripe, OpenAI, SendGrid
DB passwords	Master password, connection strings
Certificates and private keys	TLS certs, code signing, host keys
OAuth tokens	Refresh tokens
SSH keys	Private keys for server access
Cloud credentials	AWS IAM access keys, GCP service accounts
Encryption keys	Master keys for data encryption

Antipatterns - never do this

Many accidents are caused by “doing what you shouldn’t.” All below should be banned immediately, sent back if seen in review.

Antipattern	Why it’s dangerous
Hardcode in source	Permanently in Git history, hard to track or delete
Commit .env to Git	On public repos, picked up by bots in seconds
Share via Slack/email	Logs remain, leak via archives
Plaintext storage in internal wiki	Accessible to retirees and leavers
Common secrets shared by everyone	Untraceable who leaked
Output to logs	Leak via audit logs / CloudWatch

Any organization with even one of these in operations needs an immediate review. “Secrets pushed to Git” is synonymous with not “made public” but handed to bots worldwide - a perennial industry warning.

Secrets Manager - centrally managed

The modern standard is centrally managing secrets in a dedicated service. Apps fetch via API at startup, with no residue on disk or in code. Each cloud provides this as a standard service, so there’s almost no reason to build encrypted storage yourself.

flowchart TB
    subgraph BAD_WAY["Antipatterns (absolutely NG)"]
        CODE[Hardcoded in code] -.→ GIT[(Git)]
        ENV_FILE[.env committed] -.→ GIT
        GIT -.->|public repo<br/>= bot picks up in seconds| LEAK[CC charges/<br/>mining damage]
    end
    subgraph GOOD_WAY["Modern standard"]
        APP[App] -->|API at startup| SM[Secrets Manager<br/>AWS/GCP/Azure/Vault]
        SM -->|short-lived tokens| APP
        SM -.|auto-rotation| ROT[daily/weekly<br/>key updates]
    end
    classDef bad fill:#fee2e2,stroke:#dc2626;
    classDef good fill:#dcfce7,stroke:#16a34a;
    classDef sm fill:#dbeafe,stroke:#2563eb,stroke-width:2px;
    class BAD_WAY,CODE,ENV_FILE,GIT,LEAK bad;
    class GOOD_WAY,APP,ROT good;
    class SM sm;

Service	Characteristics
AWS Secrets Manager	Strong rotation automation and IAM integration
AWS Systems Manager Parameter Store	Cheap, simple, small-scale-oriented
Google Secret Manager	GCP-native integration
Azure Key Vault	Azure-integrated, also encryption-key management
HashiCorp Vault	OSS, multi-cloud, strongest features
Doppler / 1Password	Developer-oriented SaaS, polished UX

HashiCorp Vault tops in features but operations are heavy. With AWS alone, cloud-managed often suffices, and Vault is realistically considered after stepping into “multi-cloud + dynamic secrets required.”

Secret lifecycle

Secrets are managed in the lifecycle “issue → distribute → use → rotate → revoke.” Each phase has leakage risk, so countermeasures are needed in all stages.

Phase	Countermeasure
Issue	High-strength random, short-lived tokens recommended
Distribute	Encrypted communication, minimum distribution
Store	Secrets Manager, encryption
Use	Env vars, memory only, no logging
Rotation	Automatic, periodic
Revoke	Promptly, immediately on leak

Periodic rotation is insurance on the premise of “leaking someday” - 90 days is a common guideline. Done manually it’s always forgotten, so honestly using Secrets Manager’s auto-rotation is safe.

Handling in dev environments - don’t put production secrets on dev PCs

Even in development, production secrets shouldn’t be on dev PCs. Use separate secrets (mock/sandbox environment) for dev, and strictly limit production access.

Use case	Recommended handling
Local dev	.env.local + .gitignore + mock
CI/CD	GitHub Secrets, GitLab CI Variables
Staging	Secrets Manager (separated from prod)
Production	Secrets Manager + audit logs

Putting .env in .gitignore is the minimum. Ideally form the habit of putting .gitignore in first as an empty commit right after repo creation - it’s safer.

Secret Scanning - auto-detect accidental commits

The mechanism that auto-detects whether secrets are mixed into source. GitHub provides Secret Scanning as standard on all repos, with auto-notifications when major-service API keys are detected.

Tool	Use case
GitHub Secret Scanning	GitHub standard, free
GitLeaks	OSS, pre-commit-hook detection
TruffleHog	OSS, full Git-history scan
Detect-secrets	OSS by Yelp
GitGuardian	SaaS, high-feature

Running GitLeaks in pre-commit hooks detects accidental mixing at commit time. The last bastion preventing “noticing after committing” - ideally distributed to all developers.

Rotation strategy

The principle is always periodically changing secrets. Since you can’t know when leakage occurs, periodic invalidation is the only defense. Manual rotation is always forgotten, so accept that automation is required.

Secret type	Recommended frequency
DB passwords	30-90 days
API keys	30-90 days
TLS certs	60-90 days (Let’s Encrypt is auto)
OAuth refresh tokens	Per session
Encryption keys	Annually
Service accounts	30 days

AWS Secrets Manager and Vault have auto-rotation features, updating without human intervention. Design with the acceptance that “rotation that doesn’t run unless humans touch it” doesn’t exist.

Zero-secret architecture - don’t have secrets in the first place

Stepping further, designs that don’t have secrets are a strong modern option. Using IAM Role, Workload Identity, IRSA (IAM Roles for Service Accounts), distributing permanent API keys to apps becomes unneeded.

Tech	Use case
AWS IAM Role	EC2/Lambda/ECS auto-fetch temporary credentials
GCP Workload Identity	GKE, Cloud Run
Azure Managed Identity	Azure overall
IRSA	EKS (K8s Pod → AWS API)
OIDC Federation	GitHub Actions → AWS temporary auth

When deploying from GitHub Actions to AWS, the new best practice is also taking temporary auth via OIDC without API keys. Reducing permanent secrets reduces what can leak in the first place.

Handling sensitive data and PII

Apart from secrets, personal info (PII, Personally Identifiable Information) and sensitive business data also need strict management. PII leakage is also subject to regulatory fines, requiring secret-level handling.

Data type	Recommended countermeasure
Personal info (PII)	Encryption, masking, access logs
Credit-card info	PCI DSS-compliant, tokenization
My Number (Japan ID)	Legal management, strict storage
Healthcare info	HIPAA-compliant, encryption

Best practice for credit-card info is not storing in your own DB and entrusting to Stripe etc. If you don’t store, there’s nothing to leak.

Decision criterion 1: org scale

Secret-management heaviness is decided by scale. At small scale, GitHub Secrets + cloud managed services suffice; large enterprises need Vault-centric integrated platforms.

Scale	Recommended platform
Personal/small	AWS SSM Parameter Store / GitHub Secrets
Startup	AWS Secrets Manager
Mid-size	Vault or Doppler
Large enterprise	Vault Enterprise + HSM

Decision criterion 2: compliance

In industries needing regulatory response, audit logs, rotation history, and access control all need to remain as trails. A world demanding not “we do it” but “we can prove it.”

Regulation	Major requirements
SOC 2	Rotation, logs, access control
ISO 27001	Documented management process
PCI DSS	Strict key-management rules
HIPAA	PHI (Protected Health Information) protection, encryption

How to choose by case

Personal / small dev (1-5 people)

GitHub Secrets + AWS SSM Parameter Store + .env.local combination is enough. During dev: .env.local (already in .gitignore); CI/CD: GitHub Secrets; production: SSM Parameter Store. Minimum composition with near-zero cost.

Startup / mid-size SaaS

The triad of AWS Secrets Manager + OIDC Federation + GitLeaks is the front-runner. Auto-rotate with Secrets Manager, GitHub Actions → AWS goes API-key-less via OIDC, pre-commit GitLeaks check. The point is leaning toward designs that don’t hold permanent API keys as much as possible.

Multi-cloud / mid-size enterprise

Step into HashiCorp Vault + Workload Identity + dedicated ops team. Centralize secret management across AWS/GCP/Azure/on-prem, eradicate permanent secrets with Dynamic Secrets (dynamic issuance). Need to commit 1-2 dedicated people to Vault operations.

Regulated industries (finance, medical, government)

Lift up to Vault Enterprise + HSM + audit logs + segregation of duties. Protect keys with FIPS 140-2-certified HSMs, leave all secret issuance/use/changes in audit logs, and organizationally separate key administrators from users. Going this far reaches the level of passing audits.

Phased practical matrix for secret operation

Secret management is not “put in Secrets Manager and done” - it’s designing the whole lifecycle.

Phase	What to do	Automation tool	SLA
Issue	High-strength random + short-lived recommended	Vault Dynamic Secrets	-
Distribute	Encrypted communication, via Secrets Manager	AWS Secrets Manager / Vault	-
Store	KMS encryption + access control	Managed Secrets Manager	-
Use	Env vars, memory only, no logging	API fetch at app startup	-
Rotation	Automatic, periodic	Secrets Manager / Vault auto-rotation	30-90 days
Revoke	Immediately on leak	Incident-response playbook	Within 5 min
Monitor	Detect accidental commits with Secret Scanning	GitHub Secret Scanning / GitLeaks	At commit

Rotation frequency guidelines: DB passwords 30-90 days, API keys 30-90 days, TLS certs 60-90 days (Let’s Encrypt auto), OAuth refresh tokens per session, encryption keys annual, service accounts 30 days. Automation is required on the premise that manual rotation gets forgotten.

Secret lifespan is safer the shorter. Eliminate permanent keys to the limit - the direction.

Secret-management pitfalls and forbidden moves

Typical accident patterns in the secrets area. All link directly to leak = instant unauthorized access.

Forbidden move	Why it’s bad
Hardcode in source	Permanently in Git history. The cause of the Uber incident (57M leaked, $148M settlement)
Commit .env to Git	GitHub bots pick up in seconds. Instant Bitcoin-miner running
Plaintext share via Slack/email/wiki	Leaks via log archives, retirees can see
Common secrets shared by everyone	Untraceable who leaked
Output secrets to logs	Leak via audit logs, permanent in CloudWatch Logs
Push AWS Access Keys to public GitHub	Bitcoin-miner-grade large instances spin up uninvited in hours, hundreds of thousands of yen billed
Run without rotation	No insurance on the “leak someday” premise
Leave real keys in .env.example	Mistaken as test, leak surfaces 6 months later
Paste confidential code into ChatGPT like the Samsung incident	Becomes external training target; the company banned business ChatGPT use in 2023
Ignore Dependabot alerts	500 lined up on day one, notifications wallpapered, real alerts also missed in fatigue
Don’t consider going secret-less (IAM Role / OIDC)	Keep holding permanent API keys, accumulating leak risk
”If I commit .env, I just delete it from history” — complacency	Complete deletion from Git history is extremely difficult; immediate rotation required
”Internal Git is safe” — storing in plaintext	Retiree exfiltration incidents are common; no plaintext even internally
”Few developers, so common keys are OK” — compromising	Untraceable who leaked; issue individually regardless of count
”Encrypting before saving secrets is safe” — overconfidence	Meaningless if the decryption key sits plaintext nearby; KMS/Vault key-ciphertext separation required

The 2016 Uber data-leak incident (5.7M leaked + $148M settlement starting from accidentally-committed AWS credentials on GitHub) and the 2023 Samsung ChatGPT-paste incident are both typical cases of “one person’s slip-up” shaking the whole organization’s reputation.

For secrets, “the best management is not having them.” Aim for going secret-less (IAM Role / OIDC).

AI decision axes

AI-era favorable	AI-era unfavorable
Secrets Manager for centralized management	Hardcoded in source code
Secret Scanning (pre-commit detection)	No accidental-commit detection
Going secret-less (IAM Role / OIDC)	Persistent API Key operation
Filters hiding secrets from AI	Mixing confidential info into prompts

1. No hardcoding — ban source/Wiki/Slack instantly, centralize in Secrets Manager
2. Auto-rotation — auto-update in 30-90 days, no human intervention
3. Aim for going secret-less — reduce permanent keys with IAM Role / Workload Identity / OIDC Federation
4. Secret Scanning + hide from AI — GitLeaks at pre-commit, filters preventing AI mixing

Author’s note - cases where “just one accidental commit” tilted a company

Cases where accidental secret commits link directly to billions in damage have become half-perennial industry talking points.

The 2016 Uber data-leak incident exploited AWS credentials remaining on an internal engineer’s private GitHub, leaking about 57M drivers/users’ data. Uber initially concealed the fact and paid the attackers $100,000 in the name of a “bug bounty” to delay disclosure - resulting in subsequent class-action settlement on the order of $148M. A textbook case where “just one API key” damaged company reputation and finances simultaneously.

Another - in 2023, a Samsung semiconductor-division engineer pasted confidential source code into ChatGPT for consultation, putting the confidential code into a state where it could become external training material - reported as an incident, and the company banned business ChatGPT use company-wide. An AI-era-specific case where the act of “pasting into the prompt area” surfaced as a new leakage path.

Both are cases where one person’s slip-up shook the whole organization’s reputation, slapping home the importance of defending by mechanism (Secret Scanning, AI-purpose filters, going secret-less). Designs depending on individual care break down probabilistically someday - the architect’s responsibility is building mechanisms on that premise.

What to decide - what is your project’s answer?

For each of the following, try to articulate your project’s answer in 1-2 sentences. Starting work with these vague always invites later questions like “why did we decide this again?”

• Secrets Manager selection (AWS / Vault / Doppler)
• Dev-environment separation (no production secrets on dev PCs)
• Secret Scanning introduction (GitLeaks / GitHub standard)
• Rotation frequency (30-90 day standard)
• Scope of going secret-less (IAM Role, Workload Identity)
• Policy during AI development (hiding from AI)
• Incident-response procedures (immediate revocation, incident response)

Summary

This article covered secret management, including Secrets Manager, Secret Scanning, auto-rotation, going secret-less with IAM Role/OIDC, and design that doesn’t show secrets to AI.

No hardcoding, auto-rotation, aim for going secret-less, Secret Scanning + hide from AI. That is the practical answer for secret management in 2026.

Next time we’ll cover vulnerability assessment (SAST, DAST, SBOM, dependency-library management).

Back to series TOC -> ‘Architecture Crash Course for the Generative-AI Era’: How to Read This Book

I hope you’ll read the next article as well.