Precise, Stable, and Still Wrong: The Illusion of Certainty in QRA

Overview

Precise, Stable, and Still Wrong: The Illusion of Certainty in QRA

A Monte Carlo simulation can do everything we ask of it.

It can run thousands of iterations.It can converge successfully.It can produce stable P-values, sensitivity charts, and confidence levels.

And it can still produce the wrong answer.

Not because the mathematics failed.

Because the model may have precisely calculated assumptions that were never accurate enough in the first place.

This is one of the most uncomfortable realities in quantitative risk analysis.

As projects move through governance and funding cycles, uncertainty gradually transforms into:

cost and schedule confidence levels

percentile-based forecasts

stable contingency positions

increasingly confident delivery narratives

The numbers become sharper.

The outputs become cleaner.

The discussion becomes more confident.

But the underlying understanding of the project does not always improve at the same pace.

Sometimes, the sophistication of the analysis begins to exceed the maturity of the assumptions behind it.

That is when projects become precisely inaccurate.

This is not an argument against QRA or Monte Carlo simulation.

Quite the opposite.

Monte Carlo remains one of the most valuable tools available for understanding uncertainty in complex projects.

The issue is not the tool.

The issue is forgetting that simulation does not create certainty. It structures and amplifies the assumptions we provide.

Earlier articles in this series explored different ways uncertainty can be represented, distorted, and interpreted in quantitative risk analysis.

The first articles focused on improving the model itself:

choosing distributions based on risk behaviour rather than rules of thumb,

representing conditional logic, dependencies, and scenario-based risks,

avoiding simplifications that distort tail behaviour.

The discussion then moved beyond modelling mechanics:

why technically correct QRA outputs can still fail to influence decisions,

why the best-looking option in a model is not always the most resilient strategy.

This article continues that progression by exploring another challenge:

What happens when the model becomes more precise than our actual understanding of the project?

Because sophisticated outputs do not automatically mean sophisticated certainty.

Precision Is Not the Same as Accuracy

One of the quiet dangers in QRA is that precision and accuracy are often treated as though they are closely related.

They are not.

A model can:

converge properly,

generate smooth probability distributions and S-Curves,

produce stable cost and schedule confidence levels,

provide consistent sensitivity rankings and key risk drivers,

while still being materially inaccurate in how it represents real project exposure.

This is especially true in complex delivery environments where:

scope maturity evolves gradually,

interfaces remain uncertain,

productivity shifts over time,

procurement conditions change,

and systemic pressures emerge dynamically rather than predictably.

A stable level of confidence does not prove a stable understanding.

And a narrow range does not necessarily mean uncertainty has genuinely collapsed.

Sometimes it simply means uncertainty has been compressed into assumptions that feel governable.

Many Inputs Are Structured Opinions

One of the least comfortably discussed realities in QRA is that many model inputs are not empirical observations.

They are structured opinions.

That does not make them invalid.

But it does mean they carry:

judgement,

behavioural bias,

selective memory,

delivery optimism,

organisational pressure,

and social influence.

Projects rarely possess enough historical data to estimate uncertainty with true statistical confidence.

Instead, much of the modelling process relies on:

SME judgement,

workshop discussions,

delivery experience,

benchmarking,

and assumptions about future behaviour.

The problem is not that judgement exists.

The problem is that organisations often stop treating assumptions like judgement once they enter the model.

After simulation, review cycles, and governance reporting, the outputs begin acquiring the appearance of objective precision.

Monte Carlo simulation does not remove judgement.

It amplifies whatever judgement is supplied.

That amplification can be incredibly valuable when assumptions are:

challenged honestly,

benchmarked properly,

and stress-tested realistically.

But when assumptions are:

politically compressed,

behaviourally optimistic,

or weakly validated,

the model can create an appearance of confidence that the underlying assumptions do not justify.

The Quiet Compression of Uncertainty

One of the most recognisable behaviours in project risk workshops is the gradual compression of uncertainty into socially acceptable ranges.

This rarely happens through deliberate manipulation.

More often, it emerges naturally through governance pressure, delivery expectations, and organisational behaviour.

Early discussions may begin with:

broad uncertainty,

immature scope,

evolving interfaces,

uncertain productivity,

or unresolved procurement assumptions.

But over time:

worst cases start feeling "too extreme,"

upper tails become uncomfortable,

optimistic assumptions become reinforced,

and uncertainty drifts toward what feels manageable and defendable.

Not necessarily what is genuinely plausible.

This is especially visible where assumptions involve:

long-term forecasts,

evolving scope maturity,

productivity expectations,

delivery performance assumptions,

market and supply chain conditions,

or the effectiveness of planned mitigation actions.

The project becomes more numerically stable.

But not necessarily more understood.

Once compressed uncertainty enters the model, the simulation begins treating it as structured credibility.

And the outputs often inherit a level of confidence the project itself has not truly earned.

Confidence Theatre

Governance systems often reward the appearance of certainty.

Not because organisations are dishonest, but because:

funding approvals,

executive accountability,

delivery pressure,

and programme commitments

all encourage stable and defensible narratives.

Under those conditions, precise-looking outputs become psychologically powerful.

A converged P90 feels safer than:

unresolved uncertainty,

unstable assumptions,

or visibly wide ranges.

This can create a form of confidence theatre.

The project gradually develops:

cleaner forecasts,

sharper percentile confidence,

and more stable contingency positions,

while substantial uncertainty remains unresolved underneath.

Clean output distributions.

Stable confidence levels.

Consistent sensitivity rankings.

Professional dashboards and reports.

All technically valid.

But none of them confirm whether the assumptions behind the model realistically reflect how the project will behave.

Precision is not proof of accuracy.

P90 Is Not a Promise

One of the most persistent governance misunderstandings is the treatment of percentiles as deterministic commitments.

Particularly P90.

Projects often behave as though:

P90 means "safe,"

contingency has now "covered the risk,"

or uncertainty has been controlled because it has been quantified.

But percentiles do not remove uncertainty.

They describe modelled uncertainty under a particular set of assumptions.

If:

upper-tail exposure is underestimated,

risk behaviours are simplified or averaged away,

or uncomfortable assumptions are compressed into acceptable ranges,

then the percentile inherits those limitations.

A confidence level is only as credible as the assumptions and behaviours that created it.

Projects Often Know Less Than Their Models Suggest

Projects frequently operate with:

incomplete interface visibility,

evolving stakeholder dynamics,

immature scope definition,

uncertain productivity,

unstable market conditions,

and unresolved sequencing assumptions.

Yet governance reporting gradually transforms these conditions into:

stable forecasts,

narrow confidence ranges,

and apparently mature contingency positions.

The presentation of certainty advances faster than the project's actual understanding.

This is especially common in early-stage projects where delivery logic has not yet been operationally tested, yet organisations still seek:

increasingly precise forecasts,

stable confidence positions,

and narrow contingency ranges.

But uncertainty does not collapse simply because governance prefers clarity.

Complexity Is Only Valuable When It Improves Understanding

One common response to uncertainty is increasing modelling sophistication.

Sometimes that helps.

Often it does not.

More complexity does not automatically create more truth.

Highly elaborate models built on fragile assumptions can deepen false confidence rather than improve understanding.

The issue is not sophistication itself.

Sophisticated modelling can be extremely valuable when it:

improves behavioural realism,

reveals hidden exposure,

tests resilience,

or changes decisions.

But complexity becomes an illusion of maturity when it increases presentation sophistication more than genuine understanding.

Complexity is only valuable when it improves understanding or decisions.

Validation Is Often Weaker Than We Pretend

Many projects:

run simulations,

generate percentile outputs,

obtain approvals,

and move forward,

without meaningfully revisiting:

whether assumptions proved realistic,

whether tail behaviour was understated,

or whether the uncertainty representation reflected how the project actually evolved.

Models are often treated as completed analyses rather than representations that should be tested, challenged, and refined as projects evolve.

Over time, confidence can become self-reinforcing rather than evidence-based.

What Mature QRA Practice Looks Like

Good QRA is not about eliminating uncertainty.

It is about understanding uncertainty honestly enough to support better decisions.

That requires:

recognising the limits of what the model can know,

representing project behaviour realistically,

making assumptions transparent and challengeable,

and applying modelling sophistication where it genuinely improves understanding.

It also requires recognising that:

stable outputs do not guarantee credible assumptions,

precise calculations do not guarantee accurate representation,

and sophisticated models do not guarantee better understanding.

Mature practice means:

challenging assumptions aggressively,

benchmarking where possible,

stress-testing tail behaviour,

focusing on material drivers,

and communicating uncertainty honestly rather than cosmetically.

Because the purpose of QRA is not to manufacture confidence.

It is to improve understanding under uncertainty.

Closing Thought

Monte Carlo simulation is not the problem.

Used properly, it remains one of the most valuable tools available for understanding uncertainty in complex projects.

But simulation does not replace judgement.

It structures and amplifies whatever assumptions the organisation supplies.

Which means:the quality of understanding matters far more than the appearance of precision.