Your team is confident.
They're also wrong together.

The failure nobody catches

Your forecasting team is confident. They agree on the numbers. The deck goes to the board. The number is wrong. Nobody is surprised, because nobody realized the team had a shared bias to begin with.

Every team that aggregates forecasts from multiple people faces the same trap. The standard fix — push the average forecast away from 0.5 — assumes the group is anchored to a neutral prior. When the group is anchored to a biased prior, the standard fix makes things worse.

This brief explains the failure mode, the math behind the fix, and what to do about it on your team this quarter.

The setup: how aggregate forecasts are supposed to work

Suppose seven analysts at your company are forecasting the probability that a major deal closes in Q3. Each gives a probability. The natural move is to average them.

That average is sometimes called the "wisdom of crowds" effect. With enough independent forecasters, the average tends to beat most individuals. The math works because individual errors cancel out.

But the math assumes the errors are independent. When the errors are correlated — when the team shares a bias — they don't cancel. They compound.

The hidden assumption in the standard fix

For decades, the recommended fix for over-conservative aggregate forecasts has been some form of extremizing transformation: take the average forecast and push it away from 0.5 toward 0 or 1. The intuition is that an average of 0.7 from a confident group probably means the true probability is closer to 0.85.

Here's the hidden assumption: the transformation pushes the aggregate forecast away from 0.5. The 0.5 is treated as the team's prior — the implicit "I don't know" anchor. Extremize from there, you get better calibration.

This works when the team's prior really is 0.5. It fails when the team's prior is biased.

The failure mode: when the prior is biased

A biased prior is any systematic shift in what the team takes as "the default." Examples:

• An investment team that has backed five climate-tech companies in a row and now treats climate-tech as the default interesting category
• A sales team that has hit quota for six quarters and now treats deals in late stage as "essentially closed"
• A risk team that has only seen one type of threat in the last 18 months and now treats that threat as the baseline
• A medical consensus panel whose training was dominated by a specific demographic cohort

The team's average forecast will reflect this bias. The bias isn't visible in the forecasts themselves — each forecaster thinks they're being independent. The bias shows up when the team's predictions systematically miss in the same direction.

The standard extremizing fix doesn't catch this. It pushes the average further from whatever the team's anchor is, not further from what's true. If the anchor is biased, the result is more confident and more wrong.

What PRIOR is

PRIOR is a Slack-deployed agent that solves this. It joins your forecasting channel, asks each team member for two numbers — their probability and their meta-belief (their estimate of the group's average) — runs the calibration, and posts the result back to the channel with a verifiable ProofPacket.

Underneath, PRIOR runs on RIG's three-agent architecture. A Builder computes the calibration. A Verifier independently scores it. A Governance agent seals a hash-chained ProofPacket. The agent can't certify its own work. Every calibration is auditable.

The math: estimating the shared prior from meta-beliefs

If you ask each forecaster not just for their probability estimate but also for their estimate of the average forecast of the other forecasters, you get a signal about each person's personal anchor.

The signal works like this. If a forecaster believes the group average will be 0.75, but their own forecast is 0.50, you can infer that this forecaster thinks the group is overconfident. Their personal anchor is below 0.5. By collecting these meta-beliefs across the team, you can estimate the shared prior that none of them would report directly.

The transformation in the last two lines is the same kind of extremizing transform used in the standard fix, but with the estimated shared prior q̂ substituted for 0.5. The result is a forecast that's pushed away from the team's actual anchor — not from a neutral one they don't actually hold.

What the parameters α and β do

α controls the overall strength of the extremity shift. β controls how the shift scales with distance from the prior. Both are tuned against your historical outcomes. PRIOR starts with α = 0.7, β = 0.4 and adjusts based on a backtest on your team's past 50+ forecasts.

What the data shows

The original research on this technique was done on experimental prediction tasks with known shared priors. The setup: pairs of forecasters were given a biased prior, asked to forecast, and then submitted to the calibration algorithm. The results:

Brier scores (lower is better). Reported in Atanasov, Himmelstein, et al.

The headline: PRIOR performs at least as well as the naive method in the unbiased case (where both work) and substantially outperforms it whenever there's a shared bias. The naive extremization's failure under shared bias is the dramatic result — it makes calibration worse than the unmodified average.

How to install PRIOR

You don't need a vendor to evaluate the math. You can do it in a week.

Step 1: install in Slack

Visit prior.fyi/install. Click "Add to Slack". Authorize. Done. The bot joins any channel you invite it to.

Step 2: pose a forecast

In any channel with PRIOR:

Step 3: read the calibrated forecast

PRIOR posts the calibration back to the channel. Raw average, naive extreme, shared prior, PRIOR's call — side by side. A ProofPacket is sealed and stored.

Step 4: resolve and measure

When the event happens, run /prior resolve <forecast_id> <outcome>. The Verifier agent immediately scores the calibration and updates the season's Brier scoreboard.

What you inherit

PRIOR is the top of a stack. Underneath, RIG's agent runtime handles:

• Builder ≠ Verifier. The agent that computes the calibration is never the agent that approves it. Different runtime, different memory, different credentials.
• ProofPackets are mandatory. Every calibration is hash-chained. You can export every packet for compliance, internal review, or external audit.
• Drift detection is structural. Shared prior drift, forecaster calibration drift, model drift — all monitored continuously. Alerts fire on threshold breaches.
• Data sovereignty. RIG runs on a local inference fleet. Your meta-beliefs, forecasts, and outcomes stay in your deployment unless you put them in the cloud.

Pricing

PRIOR · A forecasting agent · Built on RIG agent infrastructure
Built on Atanasov, Himmelstein et al. published meta-belief theory
hello@prior.fyi · prior.fyi