PoC Design: A PoC That 'Sort of Worked' Has Failed

About this article

As the fifth installment (final) of the “Solution Architecture” category in the series “Architecture Crash Course for the Generative-AI Era,” this article explains PoC design.

PoC is investment to produce decisions - PoCs without answers are failures. This article handles pre-defined Go/No-Go criteria, period setting (within 3 months), differences from MVP, AI-PoC specifics (accuracy, hallucination rate), and weekly PoC cycles - design that doesn’t end with “so what?”

What is PoC in the first place

Think of a tasting session. Before officially adding a new dish to the menu, you prepare a small batch to verify the taste, cost, and operations. The purpose is to test small before committing to full investment — “is it really good?” “does it justify the cost?”

PoC is the IT version of a tasting session. Before investing tens of millions to hundreds of millions into serious development, you verify technical and business feasibility with a small prototype and obtain Go/No-Go decision material.

Without a PoC, jumping straight into serious development means the entire investment is wasted the moment technical impossibility is discovered. PoC is the mechanism for keeping failures small.

Why PoC is needed

Lower risk before serious development

Before serious investment of tens of millions to hundreds of millions, verifying with millions minimizes loss on failure.

Reduce uncertainty

New tech, new operations, AI usage - many elements unknowable without trying. Getting reliable info via PoC is rational.

Build basis for decisions

In scenes where convincing management requires demonstration, actually-running small prototypes are more eloquent than anything.

PoC vs prototype vs MVP

PoC, prototype, and MVP are similar but different. With different purposes, design policies differ too.

PoC is internal experiment for Go / No-Go judgment, MVP is product measuring whether value emerges in market. Confusing them breaks down design.

What to verify in a PoC, and what not to

PoC doesn’t verify everything - the iron rule is narrowing to the most uncertain parts. Define “what proven lets us proceed to serious development.”

PoCs verifying already-known things are waste. Choose only unknown / uncertain parts.

Go/No-Go judgment criteria

The most important PoC design element is the Go/No-Go judgment criteria. By pre-deciding “if this number is achieved, Go; if not, No-Go,” prevents post-PoC emotional disputes.

Starting PoC without pre-deciding judgment criteria is the worst. Disputes over “is this success or failure?” after finishing.

The other half of that is what a PoC should not try to verify.

PoC also clarifies scope not to verify. Vagueness here makes PoC bloat and become indistinguishable from serious development.

Period, regime and the gap to real development

The principle for PoC is short, with clear deadline. Realistic to keep within 3 months at most - longer means scope too wide.

The trap of PoC is continues forever without setting deadline. The iron rule is delimiting time and producing answers.

The regime is the next thing to size.

The principle for PoC is few people, short concentration. Large-scale regime can’t move and decisions delay.

The ideal is 5 or fewer, minimizing communication cost. Large-scale PoCs have management cost exceeding effect.

And the gap between a PoC and real development has to be stated in advance.

PoC working doesn’t mean serious-dev success. PoC just shows “can be done”; scale, operations, maintenance are separate issues.

PoC success != project success. Designing how to leverage PoC results in serious development is also important.

AI PoCs have specifics of their own.

AI / ML PoC need verification axes different from conventional. Beyond “running,” “produces business value” and “can maintain accuracy continuously” matter.

In LLM PoCs, “may significantly exceed expectations on one hand, accuracy drops at scale” - careful evaluation is needed.

What a PoC produces should be decided before it starts.

PoC outputs aren’t just running code - judgment documents are included too. The iron rule is leaving in form management can read.

Working code + 1-page summary is most effective for management.

When you should run a PoC

After PoC, choose from 3 options. Ending in No-Go is also fine PoC outcome.

Culture of not seeing No-Go as shame is important. Failed PoCs are successes preventing failure in serious investment. Some orgs reward failed PoCs.

The first axis is how much uncertainty there is.

The higher project uncertainty, the higher PoC value. With known tech and similar-project experience, PoC unneeded.

The second is the scale of the investment.

The bigger serious investment, the higher PoC value. Serious PoC for small projects is excessive.

By case, the shape lands like this.

Selecting new technology or comparing vendors. 1-2 week tech PoC + quantitative comparison table. Run 3 candidate companies’ products in same scenario, compare performance, usability, and cost. Concentrated 1-2 engineers, judgment criteria pre-agreed with clear thresholds for performance numbers and licensing fees.

AI and LLM projects. 1-4 week AI PoC + accuracy / cost / hallucination-rate evaluation. Verify with real-data samples from internal data, prototype with Dify / LangChain, Go judgment based on “business-usable accuracy X%+ + monthly inference cost within JPY Y.” Note risk of accuracy degradation at production scale.

Operational reform, RPA and workflow. 2-3 month business PoC + field-user participation. Select 5-10 pilot users from business departments, have them use real operations for 1 month, measure time / mistake reduction. Judgment criteria are “achieving X hours monthly reduction” and user-satisfaction score.

Large-scale core reform, above 100 million yen. Multiple PoCs in parallel + phased decision gates. Run tech PoC, data PoC, business PoC in parallel, Go/No-Go judgment meeting after each PoC, serious-dev approval only after all pass. Vendor selection also via PoC for actual capability measurement.

Numeric gates for PoC scale and duration

Note: Industry baseline values as of April 2026. Will become outdated as technology and the talent market shift, so requires periodic updates.

The iron rule for PoC is short and clear. Below are industry-standard guidelines.

PoCs over 3 months are close to serious development, the sign to review scope. 5-10% of serious investment is the PoC-budget guideline - for JPY 100M projects, JPY 5-10M PoC budget is appropriate. In the AI era, 1-week PoC has become realistic.

PoC is within 3 months, 5 or fewer, numerical criteria required. Missing this falls into PoC hell of unable-to-judge.

Pitfalls and forbidden moves

Here are the six most dangerous of the typical ways a PoC goes wrong. Every one of them ends in an equivocal “it more or less worked” report.

Stretching it out on “extending the PoC period will make it succeed” is wrong as well. A PoC that produces no answer will not produce one with more time; it needs restarting. Netflix’s “test and learn” culture — hundreds to thousands of A/B tests a year, each declaring its success condition, failure condition and duration in code up front, judged automatically on statistical significance — is the successful example of systematically eliminating PoCs that cannot decide. Go/No-Go criteria are insurance for the PoC, and insurance for the working relationships as well.

Three scenarios

If you are building solo or at a startup

Below about five million yen, going straight to real development without a PoC is generally faster. In the AI era an MVP can be out in a week, so the distinction between “a prototype for verification” and “the first production version” has effectively disappeared. Check only the genuinely unknown parts — an unfamiliar external API, or the accuracy of an AI — with a spike of a few days first.

Personal / Startup: Ship in One Month Is Correcten.senkohome.com/arch-intro-case-startup/

If you are a small or mid-size SaaS

This is the stage for having a lightweight PoC template: one to four weeks, two or three people, numeric gates mandatory. For AI and LLM work, judge Go or No-Go mechanically on three measures — accuracy, cost and hallucination rate; for operational reform, on the measured time saved by pilot users.

Small-Mid SaaS - Lean on Managed and Run with Few Peopleen.senkohome.com/arch-intro-case-saas/

If you are a large enterprise

Several PoCs — technical, data and business — run in parallel, with staged decision gates between them. Hold a Go/No-Go review after each, and approve real development only once they have all passed. For selecting external vendors too, measuring capability through a PoC is more reliable than reading a proposal.

Large-Enterprise Core: Design That Holds Up for Yearsen.senkohome.com/arch-intro-case-enterprise/

AI decision axes — The tempo of a PoC has shifted to weekly

AI shifted PoC tempo to weekly cycles

Traditional PoCs followed a cycle of “spend 3 months verifying 1 idea, then write a report.” AI-era PoCs have fundamentally changed tempo.

With AI coding tools, most PoC work — API connection verification, data-format conversion, simple UI construction — can be completed in days. Running 1 PoC per week and verifying a different idea the next week is now realistic as a weekly cycle.

This speed change has major implications — it gives teams the psychological safety of “failure just means there’s a next one.” A failed 3-month mega-PoC deals major organizational damage, but three failed 1-week mini-PoCs just accumulate learning.

However, as speed increases, pre-designing Go/No-Go criteria becomes even more important. Without clear judgment criteria, “let’s just extend one more week” repeats, and 3 months pass with no conclusion anyway.

AI supporting PoC Go/No-Go judgments

The biggest PoC failure pattern is “built it, but nobody can judge success or failure.” To prevent this, quantitative Go/No-Go criteria are set before PoC start — and AI can help design those criteria themselves.

Specifically, have AI collect benchmark information and case studies on the technology or service being verified, and understand the range of “realistically achievable performance values for this technology” before setting criteria. Instead of an unsupported “response under 100ms,” you can set achievable targets reverse-calculated from similar cases.

At the judgment phase after PoC completion too, having AI organize collected data and draft comparison tables and judgment rationale significantly compresses report-writing effort. The human role concentrates on “making the final Go or No-Go decision” and supplementing business context that AI tends to miss (political judgment, existing contract constraints, etc.).

What to decide - what is your project’s answer?

For each of the following, try to articulate your project’s answer in 1-2 sentences. Starting work with these vague always invites later questions like “why did we decide this again?”

• Verification purpose (what to prove)
• Judgment criteria (Go/No-Go numbers)
• Period (usually within 1-3 months)
• Regime (few people, clear lead)
• Verification scope (do, don’t do)
• Outputs (code, report, demo)
• Post-PoC progression (serious / stop / Pivot)

Author’s note - cases of “PoC hell” wasting a year

Cases of vague-purpose / judgment-criteria PoCs continuously exhausting orgs are repeatedly told.

A large enterprise, under the policy of “utilize generative AI in operations,” had each department launch generative AI PoCs independently, with the result of 14 PoCs running in parallel a year later, all “sort of worked” without conclusions, zero serious deployments - cases often reported. PoCs without judgment criteria become “content for report meetings” rather than success or failure, with field engineers exhausted, management losing AI expectations - falling into a vicious circle.

In contrast, Netflix’s “Test and Learn” culture is cited as a PoC-design success case. Netflix runs hundreds to thousands of feature A/B tests yearly, but pre-declares “success / failure conditions / period” in code for each test, with mechanisms automatically judging when results become statistically significant. Without waiting for human judgment, Go/No-Go gets decided mechanically, systematically erasing “unjudgable PoCs.”

Both show the truth from front and back that PoC value is producing decisions; PoCs not producing decisions are blocks of existence cost. Go/No-Go conditions are PoC insurance and human-relations insurance.

Related Articles

Summary

This article covered PoC design, including Go/No-Go criteria, period, regime, differences from MVP, AI-PoC specifics, and weekly-cycle-ization.

Pre-decide Go/No-Go, narrow to uncertain parts, cut at within 3 months, run weekly. That is the practical answer for PoC design in 2026.

And this was the final installment of the “Solution Architecture” category. Next time we’ll start a new category (Case Studies). Plan to dig into how the judgment axes learned in all categories so far combine in the field, through scale / phase-specific real-case comparisons.

Back to series TOC -> ‘Architecture Crash Course for the Generative-AI Era’: How to Read This Book

I hope you’ll read the next article as well.