[Security Architecture] Vulnerability Assessment - Building It Into CI for Daily Inspection

About this article

As the eighth installment (final) of the “Security Architecture” category in the series “Architecture Crash Course for the Generative-AI Era,” this article explains vulnerability assessment.

Vulnerability assessment is not a pre-release ceremony but continuous operation. The era of annual pentests is over - modern standard is automated inspection on every push built into CI/CD. This article covers the differences between SAST/DAST/SCA/IAST/pentests, CI/CD integration, and countermeasures against gaps in AI-generated code.

Other articles in this category

Why vulnerability assessment is needed

Gaps mix into AI-written code

AI-driven dev raised productivity, but AI-written code is often security-naive, and human-only review can’t keep up. Building in automated assessment to let machines watch machines is the modern premise.

Dependency vulnerabilities are surging

Modern apps depend on hundreds to thousands of libraries. Incidents like Log4Shell (the December 2021 Log4j zero-day) can happen tomorrow, and you’ll be hit by gaps in dependencies even with perfect own code.

Required by compliance

Certifications like SOC 2, ISMS, PCI DSS require continuous vulnerability management. Not just form - the implementation record (when, what was inspected, how it was handled) becomes the audit target.

Main assessment methods

Vulnerability assessment splits into multiple methods by “what to investigate, how.” None covers everything alone, so combination use is the premise.

flowchart LR
    DEV[Commit] --> SAST[SAST<br/>source static analysis]
    DEV --> SCA[SCA<br/>dependency vulnerabilities]
    SAST --> CI[CI/CD pipeline]
    SCA --> CI
    CI --> STG[Staging]
    STG --> DAST[DAST<br/>dynamic attack simulation]
    STG --> IAST[IAST<br/>internal monitoring during execution]
    DAST --> PROD[Production release]
    IAST --> PROD
    PROD --> PEN[Pentest<br/>1-2x/year]
    PROD --> BB[Bug bounty<br/>continuous]
    classDef dev fill:#fef3c7,stroke:#d97706;
    classDef ci fill:#dbeafe,stroke:#2563eb;
    classDef stg fill:#fae8ff,stroke:#a21caf;
    classDef prod fill:#dcfce7,stroke:#16a34a;
    class DEV,SAST,SCA dev;
    class CI,STG,DAST,IAST ci;
    class PROD,PEN,BB prod;

Method	Target	Timing
SAST	Source code (static)	At commit, in CI
DAST	Running app (dynamic)	Staging, production
SCA	Dependencies	At commit, daily
IAST	Inside running app	During test execution
Penetration testing	Comprehensive (manual)	1-2x/year, before release
Bug bounty	External hunters	Continuous

SAST - find code gaps via static analysis

SAST (Static Application Security Testing) reads source code and looks for vulnerabilities without running it. Targets things detectable from code patterns - SQL injection, XSS (Cross-Site Scripting), hardcoded secrets. Modern standard usage: run on every push in CI, comment on PRs.

Pros	Cons
Detect at commit time	Many false positives
Cover entire codebase	Misses runtime-only issues
Devs can fix immediately	Language/framework-dependent
Low learning cost	Can’t detect config mistakes

Representative tools: Semgrep, Snyk Code, SonarQube, GitHub Code Scanning (CodeQL).

An area of high “just put it in” value. False positives appear, but preventing one critical gap pays off.

DAST - try attacks on the running app

DAST (Dynamic Application Security Testing) sends attack requests to actually-running apps and watches behavior. Common to run periodically against staging - the strength is checking behavior in the actual environment.

Pros	Cons
Detect gaps in actual env	Slightly low coverage
Find config mistakes too	Long scan time
Language-independent	Test environment needed
Black-box possible	Login-required pages awkward

Representative tools: OWASP ZAP, Burp Suite, StackHawk, AWS Inspector.

DAST is suited for staging. Run carefully on production, always with prior approval.

SCA - watch over dependency vulnerabilities

SCA (Software Composition Analysis) checks whether dependencies have known vulnerabilities (CVE, Common Vulnerabilities and Exposures). Most vulnerabilities since 2024 originate in libraries, making SCA so important you could say SCA has higher ROI than self-written code.

Pros	Cons
Detect vulnerable dependencies instantly	Updates may not catch up
Auto-fix PRs available	Need to inspect transitive deps
Detect license violations too	Old libraries get stuck
Extremely easy to deploy	Few false positives

Representative tools: Dependabot (GitHub standard), Renovate, Snyk Open Source, Trivy.

The top-priority assessment to introduce. Dependabot is enabled with one config file.

The Equifax 2017 incident resulted from leaving an Apache Struts 2 patch (CVE-2017-5638) unpatched for 2 months, leaking about 147M people’s personal info, with settlements reaching about $700M (details in appendix “Critical Incident Cases”). It was a case showing the reality that even with not one line of own code vulnerable, one borrowed part can tilt a company - the case that decided SCA’s value.

IAST and RASP - monitor internals during execution

IAST (Interactive Application Security Testing) is a method where an in-app agent monitors behavior during test execution, combining the strengths of SAST and DAST. RASP (Runtime Application Self-Protection) operates in production to detect and auto-block attacks - positioned like an evolution of WAF (Web Application Firewall).

Method	Use case	Deployment difficulty
IAST	Precise detection during testing	Mid (agent-embedded)
RASP	Active defense in production	High (perf impact)

Both need agents embedded in apps, with high deployment cost, but adoption spreads in serious large-enterprise operations.

Penetration testing - comprehensive assessment from attacker view

A comprehensive assessment where specialists actually try attacks from the attacker’s view. The biggest value is finding combined gaps invisible to automated tools (business-logic flaws, authorization bugs, design issues), with going rates of pre-release, on major refactor, or annual periodic.

Type	Content	Cost target
External black box	Attack from public face	$5k-20k
Internal (authenticated)	Attack after login	$10k-50k
Red team	Comprehensive org-wide drill	$50k+
Source-included white box	Thorough check with code disclosed	$30k+

For vendor selection, presence of PCI QSA (PCI-certified security auditor) and OSCP (Offensive Security Certified Professional) holders is a guideline.

SBOM and supply chain

SBOM (Software Bill of Materials) is a list of all dependency components in an app. Since 2024, mandating has progressed in US government procurement, etc., and SBOM submission is becoming a transaction condition. Not just a tech requirement - a concept upgraded to a commercial requirement.

Use case	Content
Impact investigation on vulnerability	Instantly identify if Log4Shell is in your org
Procurement/audit response	Submit to customers/regulators
License management	Grasp things like GPL inclusion
Supply-chain tracking	Record of “parts” origins

SLSA (Supply-chain Levels for Software Artifacts) is a framework that levels build-pipeline reliability, becoming a wheel paired with SBOM in supply-chain security.

Decision criterion 1: scale and phase

Assessment thickness is decided by public scope and data sensitivity.

Phase	Minimum to do
MVP / internal trial	Dependabot + Semgrep free tier
B2C public	Above + DAST + annual pentest
B2B SaaS	Above + SBOM + SOC 2 continuous operation
Finance/medical/public	Above + quarterly pentest + IAST + SLSA support

At MVP stage, putting in IAST/RASP is excessive, but Dependabot is now common sense to put in unconditionally from day one.

Decision criterion 2: dev regime

Continuous vulnerability-assessment operation requires a regime. “Tools installed but no one’s looking” is meaningless.

Regime	Possible operation
Small (~5)	Auto tools only, weekly review
Mid (10-30)	1 dedicated security personnel
Large (30+)	Standing security team
Enterprise	CSIRT, SOC, red team

The modern current is the DevSecOps thinking (uniting development, security, operations) where developers hold first-line responsibility for security. The model “security team looks at it later” can’t keep speed.

How to choose by case

Startup / personal dev

Dependabot + GitHub Code Scanning (CodeQL free tier) + OSS Semgrep. This alone gets minimum detection. Pentest when major customers demand it; SBOM is enough with npm audit or pip-audit.

B2B SaaS (growth)

Unify SAST/SCA with Snyk or GitHub Advanced Security, DAST with StackHawk, annual pentest. SOC 2 continuous monitoring becomes required, and the operation of leaving vulnerability-management records is set up. SBOM submission to customers in CycloneDX format is standard.

Enterprise / finance

Commercial SAST (Checkmarx, Veracode) + DAST (Burp Pro) + IAST (Contrast Security) + RASP. Quarterly pentests + annual red team drills. SBOM auto-generated on all builds, aim for SLSA Level 3+. CSIRT (Computer Security Incident Response Team) and SOC (Security Operations Center) standing.

Common misconceptions

”Once before release is enough”

Dependencies update daily and gaps are born daily. Continuous assessment is the premise.

”With SAST, DAST is unneeded”

SAST sees code gaps, DAST sees config gaps. Different roles, both needed.

”With WAF, vulnerability assessment is unneeded”

WAF only blocks detectable attacks. The fundamental gap isn’t sealed - WAF as insurance and assessment that seals the gap itself are different things.

”Disable because there are too many false positives”

The right answer is properly tuning rules - disabling defeats the purpose. There are common stories of enabling Dependabot, having 500+ alerts on day one, no one responding, and a year later the notifications turned into wallpaper. The lesson: without deciding SLAs and on-call rotations the moment you deploy, even the best tools turn into mere noise.

”Without paid tools, it’s meaningless”

The OSS combination of Dependabot, Semgrep, and OWASP ZAP covers 70%. Realistic order: start with the free tier, then upgrade to paid once the regime turns.

Vulnerability-response SLAs and phased practical matrix for assessment

Vulnerability assessment is decided by post-discovery response speed. Below are industry-standard SLAs (Service Level Agreement).

Severity	Response deadline	Notification	Practical motion
Critical (RCE etc.)	Within 72 hours	Instant PagerDuty + Slack	Stop all work to respond
High	Within 1 week	Slack + create PR	Respond next sprint
Medium	Within 1 month	PR + Issue	Respond on plan
Low	Within 3 months	Issue	Respond on regular maintenance

For assessment-integration timing, SCA (Dependabot) is enabled from day one, SAST is integrated into CI on PR, DAST runs weekly in staging, SBOM is auto-generated on all builds, and pentests are at least annual (regulated industries quarterly) - the practical rule. As shown by Equifax 2017, leaving Critical patches for 2 months becomes company-survival-level damage.

Critical is within 72 hours - the rule. Patch delays link directly to company-survival risk.

Vulnerability-assessment-operation pitfalls and forbidden moves

Typical accidents in assessment operation. All have the structure of “installed” but “not operating”.

Forbidden move	Why it’s bad
Enable Dependabot and leave alerts	500 lined up turns to wallpaper, real Critical buried. SLA + on-call rotation required
Disable to ignore SAST false positives	Rule tuning is the right answer. Disabling defeats the purpose
Only annual pentests with no continuous assessment	Dependency vulnerabilities born daily. CI-integrated continuous assessment required
Don’t seal vulnerabilities because WAF detects	WAF is insurance, root countermeasure is parameterized queries on code side
Don’t generate SBOMs	Incidents like Log4Shell (December 2021) leave you in “don’t know if affected”
Leave vulnerability response to the security team	DevSecOps - developers holding first-line responsibility is the modern style
No zero-day playbook	Decision needed within 24 hours. Pre-prep required
No AI security assessment (Garak / PromptFoo)	Attacks against AI itself (prompt injection) unguarded
Don’t enable Secret Scanning	Git accidental commits happen daily, instant leak without detection
Ignore npm audit / pip-audit warnings	Carry known vulnerabilities into production

The September 2017 Equifax info-leak (Apache Struts 2 CVE-2017-5638 patch left for 2 months, 147M leaked, settlements about $700M), the December 2021 Log4Shell (Java apps worldwide hit zero-day overnight, companies in droves not knowing if affected without SBOM) - cases where “dependency gaps” become lethal are repeated.

Vulnerability assessment is premised on operations of building into CI/CD for daily inspection. Annual pentest alone can’t seal gaps.

AI-era perspective

When AI-driven development (vibe coding) and AI usage are the premise, vulnerability assessment enters the new cycle of AI inspecting AI-written code. CodeQL and Semgrep already introduce AI-based rule suggestions, and tools like GitHub Copilot Autofix that perform detection and fixing simultaneously are mainstreaming.

Favored in the AI era	Disfavored in the AI era
CI/CD-integrated automated assessment	Manual annual assessment only
SBOM auto-generation	Vague dependency management
Autofix-supporting tools	Detection only, neglected
Type-safe languages	Dynamic-language ambiguity

On the other hand, the modern reality is that AI itself becomes a new attack target. Areas legacy assessment tools can’t cover are emerging - prompt injection, LLM info leaks, AI agent permission escapes - and AI-dedicated security inspection (Garak, PromptFoo, etc.) becomes a new pillar.

In the AI era, attacks on AI itself are also assessment targets. SAST/DAST alone are insufficient.

Author’s note - cases where both “patch delay” and “zero-day” became lethal

Cases where the cost of skimping vulnerability assessment tilted companies are perennial security-industry talking points.

The Equifax 2017 info-leak (Apache Struts 2 CVE-2017-5638 patch left for 2 months → 147M leaked → settlements about $700M) is the case that decided SCA’s (dependency assessment) value (details in appendix “Critical Incident Cases”).

Log4Shell 2021 (Log4j’s CVE-2021-44228) overnight exposed Java-built apps worldwide to a zero-day, frequently leaving organizations unable to even know if affected without SBOM - the trigger for supply-chain security (SBOM/SLSA) being baked into contract conditions.

Both left the lesson that “dependency gaps” are lethal, and own-code SAST/DAST alone can’t fully defend. The modern minimum line is inspecting daily-updated vulnerabilities daily. We’ve entered an era where the operation of humans looking once a year no longer holds.

What to decide - what is your project’s answer?

For each of the following, try to articulate your project’s answer in 1-2 sentences. Starting work with these vague always invites later questions like “why did we decide this again?”

• SCA introduction (Dependabot / Renovate / Snyk)
• SAST introduction (Semgrep / CodeQL / SonarQube)
• DAST introduction (OWASP ZAP / StackHawk)
• SBOM-generation automation
• Pentest frequency and scope
• Vulnerability-response SLA (within how many days for Critical)
• AI security inspection (when LLM is used)

How to make the final call

The core of vulnerability assessment is the thinking that it’s not a pre-release ceremony but continuous operation built into CI/CD. AI-written code easily mixes in gaps, and dependency vulnerabilities are born daily. No matter how robust the auth/authorization design, you must face the reality that one SQL injection collapses everything. The top priority to put in is SCA (Dependabot) - the modern common sense that gaps in dependencies attract more attacks than own code. Layer SAST/DAST there, and stack IAST/pentest by scale - the standard.

Another decisive axis is the demand for security operations in the era where AI writes and AI defends. While tools like CodeQL and Copilot Autofix that detect and fix simultaneously become standard, AI itself becomes an attack target. Eyes need to turn to areas legacy tools can’t cover - prompt injection, LLM info leaks, agent permission escapes. SBOM and SLSA become baked into commercial flow as required conditions for supply-chain countermeasures, going beyond technical decisions to contract conditions.

Selection priorities

1. Enable Dependabot from day one - SCA has the highest ROI
2. Integrate SAST in CI - automated inspection on every push is the modern minimum
3. Run DAST weekly in staging - seal gaps from config mistakes
4. Auto-generate SBOM and define vulnerability SLA - becoming a commercial requirement

“Build into CI/CD and inspect continuously” is the rule. Annual pentest alone can’t seal gaps fully.

Summary

This article covered vulnerability assessment, including the use of SAST/DAST/SCA/IAST/pentests/SBOM, response SLAs, and new AI-era attack surfaces.

Enable Dependabot from day one, integrate SAST in CI, run DAST weekly in staging, auto-generate SBOM and define SLA. That is the practical answer for vulnerability assessment in 2026.

And this was the final installment of the “Security Architecture” category. Next time we’ll start a new category (Development and Operations Design).

Back to series TOC -> ‘Architecture Crash Course for the Generative-AI Era’: How to Read This Book

I hope you’ll read the next article as well.