Quant Trader
Prediction Markets vs Reality — Agent Tracks Calibration Over Time
Key Takeaway
The agent tracks prediction market calibration across Polymarket and Kalshi — found markets well-calibrated in the 50-80% range but systematically mispriced at the extremes, revealing exploitable edges.
The Problem
Prediction markets are supposed to be efficient. The crowd is supposed to be wise. Prices are supposed to reflect true probabilities.
Except they don't. Not always.
If you bet on prediction markets without understanding where they're well-calibrated and where they're not, you're gambling. You're paying a vig on markets that are efficiently priced and missing the edges that actually exist.
I wanted data. Not theory. Not "markets are efficient" hand-waving. I wanted to know: when Polymarket says something is 85% likely, how often does it actually happen? When Kalshi prices an event at 15%, does it resolve yes 15% of the time? Or 8%? Or 25%?
The answer matters enormously. If markets systematically overstate probabilities above 85%, there's a structural short. If they understate probabilities below 20%, there's a structural long. But you can only see this with a large dataset tracked over time.
The Solution
The Argus Edge skill combines prediction market data collection with calibration analysis. It tracks contract prices at various points before resolution, then measures actual outcomes against predicted probabilities. Over time, this builds a calibration curve that reveals where markets are efficient and where they're not.
The Process
The calibration tracker runs continuously:
yamlShow code
# calibration-tracker-config.yaml
markets:
polymarket:
api: "polymarket_v2"
categories: ["politics", "crypto", "tech", "economics", "sports"]
min_volume: 50000 # Only track liquid markets
snapshot_frequency: "6h"
kalshi:
api: "kalshi_v1"
categories: ["economics", "weather", "tech", "politics"]
min_volume: 10000
snapshot_frequency: "6h"
calibration:
bucket_size: 5 # 5% buckets (0-5%, 5-10%, etc.)
min_samples_per_bucket: 30
recalculate: "weekly"
tracking:
capture_price_at: ["resolution-7d", "resolution-3d", "resolution-1d", "resolution-1h"]
track_volume_profile: true
track_category_performance: true
The calibration analysis runs weekly:
pythonShow code
# Calibration calculation
resolved_markets = get_resolved_markets(lookback_days=365)
# 2,847 resolved markets across both platforms
calibration = {}
for bucket_start in range(0, 100, 5):
bucket_end = bucket_start + 5
markets_in_bucket = [m for m in resolved_markets
if bucket_start <= m.final_price < bucket_end]
if len(markets_in_bucket) >= 30:
actual_yes_rate = sum(1 for m in markets_in_bucket
if m.resolved_yes) / len(markets_in_bucket)
calibration[f"{bucket_start}-{bucket_end}%"] = {
"predicted": (bucket_start + bucket_end) / 2,
"actual": actual_yes_rate * 100,
"n": len(markets_in_bucket),
"edge": actual_yes_rate * 100 - (bucket_start + bucket_end) / 2
}
The output:
View details
PREDICTION MARKET CALIBRATION — 12-MONTH ROLLING
(2,847 resolved markets, Polymarket + Kalshi combined)
Predicted vs Actual Resolution Rate:
┌────────────┬───────────┬────────┬──────────┬───────────┐
│ Price Range│ Predicted │ Actual │ Edge │ Samples │
├────────────┼───────────┼────────┼──────────┼───────────┤
│ 0-10% │ 5% │ 8.2% │ +3.2% ✨ │ 142 │
│ 10-20% │ 15% │ 18.7% │ +3.7% ✨ │ 198 │
│ 20-30% │ 25% │ 26.1% │ +1.1% │ 234 │
│ 30-40% │ 35% │ 34.8% │ -0.2% │ 267 │
│ 40-50% │ 45% │ 44.2% │ -0.8% │ 312 │
│ 50-60% │ 55% │ 55.8% │ +0.8% │ 298 │
│ 60-70% │ 65% │ 64.1% │ -0.9% │ 276 │
│ 70-80% │ 75% │ 73.8% │ -1.2% │ 245 │
│ 80-90% │ 85% │ 79.4% │ -5.6% ✨ │ 187 │
│ 90-100% │ 95% │ 86.1% │ -8.9% ✨ │ 134 │
└────────────┴───────────┴────────┴──────────┴───────────┘
✨ = Statistically significant edge (>3%)
Key Finding: "Favorite-Longshot Bias"
Markets OVERSTATE high-probability events.
When the market says 95%, reality is ~86%.
When the market says 85%, reality is ~79%.
Markets UNDERSTATE low-probability events.
When the market says 5%, reality is ~8%.
When the market says 15%, reality is ~19%.
The 50-80% range is well-calibrated (within ±1.5%).
Category breakdown reveals further edges:
View details
CALIBRATION BY CATEGORY (at 85%+ price point):
┌──────────────┬───────────┬────────┬──────────┐
│ Category │ Predicted │ Actual │ Edge │
├──────────────┼───────────┼────────┼──────────┤
│ Politics │ 85%+ │ 78% │ -7% ✨ │
│ Crypto │ 85%+ │ 81% │ -4% ✨ │
│ Economics │ 85%+ │ 83% │ -2% │
│ Tech/Product │ 85%+ │ 76% │ -9% ✨ │
│ Sports │ 85%+ │ 84% │ -1% │
└──────────────┴───────────┴────────┴──────────┘
Politics and tech/product are the most overconfident
categories at high probability levels.
The Results
2,847 (resolved, 12-month window)
Markets tracked
50-80% (within ±1.5%)
Well-calibrated range
80-100% (actual 5-9% lower than priced)
Overconfidence zone
0-20% (actual 3-4% higher than priced)
Underconfidence zone
Tech/product at high probabilities
Most mispriced category
Sports (bookmaker expertise effect)
Best-calibrated category
12
Monthly calibration reports generated
67% win rate on tail trades
Profitable trades from calibration insights
The favorite-longshot bias is well-documented in traditional betting markets. What's interesting is that it persists in prediction markets, which are supposedly more "rational" than sportsbooks. The explanation is behavioral: people overpay for certainty. When something looks 90% likely, the emotional cost of being wrong on a "sure thing" makes people bid it higher than warranted.
The practical implication: systematically selling "NO" contracts priced above 85% has positive expected value. Not on every market — volume, liquidity, and category matter — but as a class, these are structurally overpriced.
Try It Yourself
Install the Argus Edge skill. You need API access to Polymarket and/or Kalshi. The calibration analysis requires at least 6 months of resolved market data to produce statistically significant results. The agent starts collecting from day one, but don't trade on calibration insights until you have 1,000+ resolved markets in your dataset.
Focus on categories you understand. The edge is real, but execution matters — thin markets have wide spreads that eat your edge.
The crowd is wise. But at the extremes, it's confidently wrong. That's where the money is.
Related case studies
Quant Trader
Kalshi vs Polymarket on the Same Event — The Agent Found a 7% Edge
AI agent monitors prediction market pricing across Kalshi and Polymarket, detecting a 7% cross-platform arbitrage spread for risk-free profit opportunities.
Portfolio Manager
Polymarket Odds Shifted 15% Overnight — My Agent Woke Me Up
AI agent monitors Polymarket for probability shifts above 10%, cross-references news and on-chain data, and detects arbitrage opportunities across correlated prediction markets.
Portfolio Manager
The Complete Financial Operating System — All 17 Skills in One Stack
How 17 AI finance skills work as one integrated system: data ingestion, analysis, execution, and reporting. 31 agents across 8 teams running a complete financial operating system on Mr.Chief.
Want results like these?
Start free with your own AI team. No credit card required.