The calibration ledgers — forecast and pWin

A prediction with no track record is an opinion. FedGrade scores its own predictions against realized outcomes using proper scoring (Brier score plus the reliability/resolution decomposition), and publishes — or privately reports — the result. There are two ledgers, computed from the same scoring primitive but governed by opposite privacy stances.

The forecast calibration ledger (public)

The forecast track record is the moat claim, so it ships as a public API and a public page. It scores library-wide Forum Forecast predictions: for resolved predictions, how well did the predicted probabilities match what actually materialized?

• Metrics: Brier score, base rate, the reliability/resolution decomposition, and resolved / materialized / missed / pending counts, for both a rolling 90-day window and all-time.
• Endpoint: GET /api/trust/forecast-calibration — no auth, CDN-cached, aggregate counts and metrics only. No row contents, no tenant data, no workspace ids.
• Also exposed as the get_forecast_calibration MCP tool.
• Honesty contract: when no predictions have resolved yet, the status is "pre-launch" and the windows carry zero counts with null metrics. No invented numbers — live calibration begins only once predictions resolve against outcomes.

This is the same surface as the Forecast materialization log: if the forecaster is wrong, the ledger shows the misses.

The pWin calibration ledger (tenant-private)

The pWin ledger is the opposite stance. A pWin assessment is competitive bid intelligence, so its calibration is tenant-private — computed per-workspace, never pooled with another workspace's, and never published.

• For each model version, it scores predicted win-probabilities (score / 100) against realized won/lost outcomes.
• Abstentions are excluded honestly: no-bid and unresolved pursuits, and assessments the engine graded insufficient-evidence or recommended no-decision, don't belong in a calibration of the engine's confident calls.
• Grouped by model version (weights change across versions, so they're never pooled) and deduped on the inputs hash so identical re-runs count once.
• Endpoint: GET /api/pwin/calibration — Team+, workspace-scoped.
• Honesty contract: pre-data it ships correct-but-empty (configured: true with no model-version rows) until outcomes resolve — never a fabricated Brier on no data.

Why two stances

Public forecast calibration is a trust signal anyone can audit. Private pWin calibration is your own model-quality readout on your own pursuits — sharing it would leak your bid posture. Same math, different exposure, both honest about empty data.