You fix your #1 failure mode. Yield goes up. Everyone's happy for a month.

Then the next Pareto report comes in, and there's a new #1 — something that barely registered before. It looks like a new problem appeared out of nowhere. Engineering calls it whack-a-mole. It isn't. It's math.

The Chart Was Never Measuring What You Thought It Was Measuring

Most test sequences stop on first failure. It's the sane default — why keep running tests on a unit you already know is bad? But it has a quiet side effect: any test downstream of the failing test never runs. That test's true pass/fail outcome for that unit is unknown. Not zero failures — unknown.

Your Pareto chart doesn't distinguish between “this test passed” and “this test never got a chance to run.” Both show up as “not a failure” in the count. So the failure rate you're looking at for every test past your #1 bottleneck isn't a failure rate at all — it's a failure rate conditioned on surviving everything upstream.

Statisticians have a name for this: censored data. It's the same problem reliability engineers deal with when a study ends before every unit has failed — you know some units survived at least that long, but you don't know what would have happened next. Your test sequence does the same thing to every station after the first bottleneck.

Fix the Bottleneck, and the Population Changes Underneath You

When you eliminate failure mode #1, units that used to die early now flow through to test #2, #3, #4. Their true failure rates — which were always there, just hidden — become visible for the first time. The new #1 on next month's Pareto chart isn't a new defect introduced by the fix. It's a defect that was present in the population all along, sitting behind a station your test sequence never let it reach.

This matters for more than the Pareto ranking. It matters for SPC.

If you're calculating Cpk on a parametric measurement taken only after some earlier gate, your sample is silently conditioned on passing that gate. If the thing causing early failures — a bad component lot, a marginal solder process, a fixture drift — also correlates with the parameter you're measuring downstream, your capability numbers are biased before you've run a single calculation. You're not looking at your process. You're looking at the subset of your process that survived an unrelated filter.

This Is Architectural Debt, Not a Reporting Glitch

It's tempting to treat this as an analytics problem — a Pareto chart that needs a footnote, a Cpk calculation that needs a caveat. It isn't. It's architectural debt, the same species covered elsewhere in this series: a decision made once, for good reasons, at the test sequence level — stop on first failure, save cycle time — that quietly accrues cost somewhere downstream where nobody's watching for it.

Nobody decided to bias your capability studies. Nobody decided to hide failure modes from the Pareto chart. Those were never decisions at all — they were side effects of a test-execution choice made for throughput, inherited by every analytics tool built on top of that data without anyone re-examining the assumption. That's the pattern architectural debt always follows: the interest doesn't show up on the statement that recorded the original decision. It shows up two systems downstream, in a chart that looks statistically sound and isn't.

The Other Failure Mode: Test-to-Completion Isn't Free Either

The obvious fix is to stop stopping — run every test regardless of upstream failures, log everything, let the Pareto chart see the whole population. This does solve the censoring problem. But it introduces a different one: cascade failures.

If a power supply test fails, every downstream test that depends on that voltage rail is going to fail too — not because those subsystems are actually broken, but because they never had valid conditions to test under. Now your Pareto chart is full of correlated symptoms of one root cause, all competing for attention as if they were independent problems. You've traded a masking problem for a noise problem.

What Actually Needs to Happen

Neither stop-on-fail nor test-to-completion is “correct” in isolation. The fix is architectural, not procedural:

• Track “reached” as its own data point, separate from pass/fail. A test that never ran isn't a pass. Knowing your denominator per station — not just per unit — is what turns a misleading Pareto chart into an honest one.
• Design test blocks with real independence where possible, so a failure in one subsystem doesn't manufacture false positives three stations downstream.
• Condition your Cpk analysis on the right population. If you're calculating capability on a parameter that only a subset of units ever reached, that needs to be visible in the analysis, not buried in the way the data was collected.

This is exactly the kind of thing that's invisible until your data architecture captures it — and invisible forever if it doesn't. A Pareto chart built on step-level data that distinguishes “failed,” “passed,” and “never reached” tells a fundamentally different story than one built on pass/fail alone. Most legacy test loggers were never designed to make that distinction. It's worth checking whether yours does.