AI Probability Engine
3-model Bayesian consensus: AI + Heuristic + Microstructure
Predite's AI probability engine is the brain behind the scanner. It estimates true event probabilities for prediction market contracts, which is then compared to market prices to identify edge. This document explains how it works, what it's good at, and what it isn't.
The 3-Model Consensus Approach
We don't rely on a single AI model. Instead, three independent estimation methods run and their outputs are weighted into a final probability:
Model 1: LLM Analysis (50% weight) A large language model (currently Claude) is given the market itself and asked for a probability. What goes into the prompt:
- •The market question and its description
- •Its category
- •The current YES price, volume and liquidity
- •Days until resolution
That is the whole input. No news articles, no polls, no external feed of any kind is fetched or attached — the model reasons from the question and the market's own numbers, plus whatever it already knows. Treat it as a well-read second opinion on the wording and the price, not as something that has read this morning's headlines.
This model only runs when an LLM key is configured. When it isn't, it is skipped and the other two are reweighted to 60/40 — worth knowing, because it changes what the number means.
Output: probability estimate with structured reasoning.
Model 2: Heuristic Engine (30% weight, 60% with no LLM) A rules-based system with no access to price history. It reads four things off the current snapshot:
- •Liquidity band — thinner books are more likely to be mispriced
- •Volume ÷ liquidity ratio — heavy turnover on a thin book suggests informed flow
- •Time to resolution — the further out, the more room the estimate is given
- •Favorite-longshot correction — markets tend to overprice longshots and underprice heavy favorites
Only the last of those has a direction. The first three set *how far* the estimate may move; the favorite-longshot term decides *which way*. On a market priced between roughly 30% and 70%, where that bias doesn't apply, this model returns the market price and contributes no edge — deliberately, because none of its inputs can tell YES from NO there.
Output: numerical estimate without natural language reasoning.
Model 3: Market Microstructure (20% weight, 40% with no LLM) This one looks at the shape of the quote rather than the subject matter. It uses three inputs: the implied spread between the YES and NO prices, traded volume, and hours until resolution. A wide spread nudges the estimate away from the market price; heavy volume (above $5M) shrinks that nudge, on the view that a busy market is an efficient one; thin volume (under $50k) widens it; and inside the last 24 hours the whole adjustment is damped, because there is little time left for a repricing. There is no table of historical base rates anywhere in this engine.
Why Three Models, Not One
Each model has known failure modes:
- •LLM: can hallucinate confidently, may anchor on recent news too heavily
- •Heuristic: rigid, can't handle novel situations or contextual nuance
- •Anchor: useless for unique events without historical precedent
By combining them, errors tend to cancel out. The final estimate is more robust than any individual model.
Calibration
A probability estimate of "70% likely" should mean that, across many such predictions, the event actually happens 70% of the time. This is called calibration.
We track calibration internally, though more coarsely than a full reliability curve:
- •Each prediction is logged with its timestamp, estimate and eventual outcome
- •Over the last 30 days of resolved signals we compute two numbers: the overall hit rate, and the hit rate restricted to signals with an edge of 10pp or more
- •The second number is the one that matters — it answers "when the model was loud, was it right?"
There is no bucketing into 5% probability bands today, so we cannot yet show you a reliability curve of predicted-versus-actual across the whole range.
Current calibration (as of mid-2026):
- •Politics: well-calibrated within 3pp across most ranges
- •Sports: slight optimism bias (predicted 70%, actually 67%)
- •Crypto: higher variance, less consistent
- •Economic: well-calibrated on Kalshi-style indicators
Calibration is published periodically in our blog. It's the most honest measure of AI accuracy.
What the AI Is Good At
Established event types: elections, sports, economic indicators. The AI has lots of training data and clear patterns.
Markets with public information: news-driven outcomes where the relevant data is freely available.
Quantitative questions: "Will X exceed Y by Z date" with clear thresholds.
Aggregated reasoning: combining multiple sources into a single estimate.
What the AI Struggles With
Insider-driven markets: governance votes, M&A speculation, regulatory decisions. People with private information have edge AI can't replicate.
Cultural events: Oscars, virality predictions, fashion. AI doesn't have good models of taste.
Very recent breaking news: training data has lag. Major events from the last 24h may not be reflected.
Long-tail outcomes: rare events that don't have historical precedent.
Manipulation-sensitive markets: low-volume markets where one trader can move the price.
For these categories, treat AI estimates with skepticism and rely more on your own domain knowledge.
How News Integration Works
We continuously pull headlines and key facts from:
- •General news APIs (for political and economic context)
- •Sports data services (for game-specific markets)
- •Crypto-specific feeds (for token/protocol markets)
- •Resolution-relevant primary sources
These get fed into the AI's context window when analyzing related markets. So if there's a major news event affecting a market, the AI's analysis reflects it.
Caveat: news integration has limits. We can't cover every news source. We can't read paywalled content. We can't anticipate stories that haven't broken yet.
How AI Confidence Is Computed
The confidence score (0-100%) is the weighted average of the confidence each model reports about itself — the same 50/30/20 weights used for the probability. That has one consequence worth internalising: there is no agreement term. Three models that flatly contradict each other produce exactly the same headline confidence as three that agree, as long as each is individually sure. A high confidence means "each model was confident", not "the models converged".
What moves each model's own confidence: for the heuristic, the size of the edge it found; for the microstructure model, a wide spread, high volume, or being close to resolution. Distance from 50% is not an input to any of them.
A 90% confidence estimate is genuinely more reliable than a 50% confidence one. Trust the confidence metric — it's not marketing fluff.
Common Mistakes Users Make
Trusting AI estimates blindly: AI is wrong sometimes. Significant losses come from over-confident bet sizing on AI signals.
Ignoring confidence: a 10pp edge at 40% confidence is worse than a 5pp edge at 90% confidence. Account for both.
Trading every signal: AI shows hundreds of small edges. Most aren't worth the friction. Filter for >5pp edge + >70% confidence as a baseline.
Anchoring on AI's number: if AI says 70% and market is 60%, your job is to evaluate WHETHER the AI is more right than the market. Don't just copy the AI estimate as your view.
Improving the AI
We iterate continuously:
- •Retraining the LLM context with new event types
- •Tuning heuristic parameters based on backtest performance
- •Adjusting weight ratios based on observed calibration
- •Adding new news sources as we find them
Major changes are announced in our changelog. If you notice systematic AI errors in specific market categories, report them via the in-app feedback (it actually goes to a human and we read it).
Use Cases by Plan
Starter plan: AI estimates in the scanner, read-only — see signals and learn the platform. Venue coverage is not part of the plan: every plan's scan fetches and analyses Polymarket *and* Kalshi, and the POLY/KLSH chips just filter what is already there.
Pro plan: AI estimates across Polymarket and Kalshi. Plus arbitrage signals (AI compares prices across platforms).
Bot plan: AI estimates feeding into automated bot strategies. The EV Follower template uses these directly.
Honest Limitations
We don't sell AI as magic. Real performance:
- •Average edge identified: 2-5pp (lower than scanner displays raw)
- •Win rate on AI signals: 55-60% (slightly above random)
- •After execution costs: positive expected value, but not large
- •Variance is real: drawdowns happen, sometimes 20-30% before recovery
This is consistent with the underlying truth: prediction markets are mostly efficient, and informational edge in the era of AI is real but modest. We're not promising 30% monthly returns. We're providing tools to extract modest edge consistently over years.