Should you act on this?
Not on this evidence. 4983 order-2+ calls across 1556 stories, which share events and instruments and collapse to 65 independent bets, hit rate 41% (95% CI 40–43%), average -0.04% per call with t = -0.5 against a bar of 2.04 at 64 degrees of freedom. That is not distinguishable from luck. Use the chains as research prompts, not as signals.
How to read every table below: hit rate is the share of calls where the excess return over the benchmark reached at least a third of the expected move for that instrument's volatility. avg vs benchmark is the excess return with the call's direction applied — the number you would have made per call following every one of them, before costs. bets is the real sample size: calls that share an event or an instrument are one bet, counted once — a Saudi pipeline attack arriving as eighteen headlines and resolving to the same crude contracts, refiners, tankers and airlines is one position, not eighteen tests of anything. t is measured across those bets, and hovering it shows the bar it has to clear at the degrees of freedom it actually had; that bar is 2.0 only when the bets column is large. Brier scores the stated confidence (0 perfect, 0.25 is what always saying 50% gets). Rows under 20 calls or 10 independent bets are greyed and prove nothing.
At the horizon each call was made for, by chain order
The product's own falsification test. The thesis is that order-2/3 effects carry the edge because they take days to weeks to propagate. If they do not beat order-1 here, the thesis is wrong.
| order | n | bets | hit rate | 95% CI | avg vs bench | median | t | brier |
|---|---|---|---|---|---|---|---|---|
| 1 | 1980 | 771 | 36% | 34–38% | -0.43% | +0.06% | -0.7 | 0.409 |
| 2 | 3295 | 134 | 39% | 38–41% | -0.15% | +0.10% | -0.2 | 0.277 |
| 3 | 1308 | 173 | 45% | 43–48% | +0.21% | +0.56% | -0.4 | 0.258 |
| 4 | 380 | 94 | 44% | 40–50% | +0.07% | +0.44% | 0.6 | 0.251 |
Calibration: does stated confidence mean anything?
A calibrated model's 0.6 calls hit about 60% of the time. If every bucket hits at the same rate, confidence carries no information and should be ignored when sizing.
| stated confidence | n | bets | hit rate | 95% CI | avg vs bench | median | t | brier |
|---|---|---|---|---|---|---|---|---|
| < 0.40 | 189 | 90 | 45% | 38–52% | +0.69% | +0.17% | 0.5 | 0.267 |
| 0.40 – 0.55 | 2625 | 85 | 44% | 42–46% | -0.06% | +0.19% | -1.5 | 0.249 |
| 0.55 – 0.70 | 2483 | 299 | 39% | 37–41% | -0.23% | +0.20% | -0.6 | 0.281 |
| ≥ 0.70 | 1666 | 611 | 34% | 32–37% | -0.26% | +0.03% | 0.3 | 0.454 |
Order-2+ calls by trend agreement at entry
Every call is stamped with whether it ran with or against the instrument's own 12-month momentum when it was made. If calls against the trend hit less often, the score's trend down-weighting is earning its place; if they hit just as often, it should be removed.
| trend at entry | n | bets | hit rate | 95% CI | avg vs bench | median | t | brier |
|---|---|---|---|---|---|---|---|---|
| with trend | 1841 | 139 | 35% | 33–37% | -1.25% | -0.50% | -1.4 | 0.280 |
| against trend | 1520 | 162 | 44% | 42–47% | +0.62% | +0.54% | 0.8 | 0.267 |
| no trend / unknown | 1622 | 149 | 46% | 44–48% | +0.71% | +0.73% | 1.2 | 0.263 |
By market
| market | n | bets | hit rate | 95% CI | avg vs bench | t |
|---|---|---|---|---|---|---|
| us | 4288 | 230 | 41% | 40–42% | -0.23% | 2.2 |
| india | 1118 | 63 | 43% | 40–46% | +0.24% | -0.6 |
| commodity | 929 | 5 | 29%* | 26–32% | -0.49% | -0.7 |
| global | 536 | 1 | 45%* | 41–49% | +0.25% | — |
| crypto | 92 | 3 | 35%* | 26–45% | +0.14% | 0.1 |
By event type
| event type | n | bets | hit rate | 95% CI | avg vs bench | t |
|---|---|---|---|---|---|---|
| m and a | 1187 | 84 | 39% | 36–41% | -0.11% | -1.3 |
| supply disruption | 1183 | 7 | 35%* | 32–38% | -0.36% | 1.4 |
| earnings | 915 | 174 | 42% | 38–45% | -0.08% | 1.1 |
| geopolitics | 873 | 6 | 40%* | 36–43% | -0.05% | 0.3 |
| policy rates | 547 | 8 | 48%* | 44–52% | +0.79% | 0.3 |
| guidance | 532 | 98 | 41% | 37–46% | -0.12% | 0.7 |
| product tech | 408 | 20 | 42% | 38–47% | +0.05% | -0.6 |
| regulation | 355 | 29 | 44% | 39–49% | +0.65% | 0.6 |
| commodity move | 271 | 2 | 32%* | 27–38% | -0.53% | -1.3 |
| credit default | 212 | 30 | 44% | 37–51% | -2.57% | 1.5 |
| macro data | 184 | 8 | 37%* | 30–44% | +0.04% | -1.5 |
| other | 155 | 17 | 32% | 25–40% | -1.30% | -1.7 |
| litigation | 88 | 18 | 45% | 35–56% | -0.77% | -0.3 |
| disaster | 53 | 8 | 47%* | 34–60% | -0.71% | 0.1 |
Every order × every horizon
All four fixed horizons regardless of what the call asked for. Useful for seeing whether an effect shows up early and fades, or builds.
| order | horizon | n | bets | hit rate | 95% CI | avg vs bench | median | t | brier |
|---|---|---|---|---|---|---|---|---|---|
| 1 | 1h | 2254 | 809 | 30% | 28–32% | -0.02% | -0.01% | -0.1 | 0.431 |
| 1 | 1d | 2210 | 795 | 35% | 33–37% | +0.05% | +0.10% | 1.0 | 0.408 |
| 1 | 1w | 1969 | 703 | 37% | 35–39% | -0.34% | +0.23% | -0.4 | 0.394 |
| 1 | 1m | 1118 | 376 | 35% | 32–37% | -1.41% | -0.56% | -1.4 | 0.405 |
| 2 | 1h | 5287 | 121 | 36% | 35–37% | -0.03% | -0.01% | -1.4 | 0.274 |
| 2 | 1d | 5146 | 122 | 38% | 37–40% | +0.01% | +0.03% | -0.2 | 0.275 |
| 2 | 1w | 4656 | 122 | 38% | 37–40% | -0.21% | -0.12% | -0.5 | 0.276 |
| 2 | 1m | 2720 | 96 | 39% | 37–41% | -0.14% | +0.08% | -0.5 | 0.278 |
| 3 | 1h | 2719 | 207 | 41% | 39–42% | +0.05% | +0.01% | 2.2 | 0.253 |
| 3 | 1d | 2645 | 212 | 42% | 40–44% | +0.11% | +0.07% | 0.1 | 0.255 |
| 3 | 1w | 2333 | 208 | 43% | 41–45% | +0.26% | +0.10% | 1.1 | 0.255 |
| 3 | 1m | 1225 | 169 | 45% | 42–47% | +0.34% | +0.58% | -0.3 | 0.258 |
| 4 | 1h | 774 | 121 | 39% | 36–43% | -0.02% | +0.01% | -1.4 | 0.246 |
| 4 | 1d | 757 | 121 | 39% | 35–42% | -0.05% | -0.02% | 0.8 | 0.245 |
| 4 | 1w | 687 | 116 | 45% | 41–49% | -0.01% | +0.19% | 0.8 | 0.252 |
| 4 | 1m | 351 | 90 | 44% | 39–49% | -0.10% | +0.53% | 0.5 | 0.247 |
240 prediction(s) still open, 233 waiting for a first traded price. 1040 call(s) are on instruments with no benchmark — FX pairs, the dollar index, the 10-year — so for those the “excess” return is the raw return, because there is no index to subtract.
Recently graded
† graded at a fixed horizon other than the one the call was made for. Hover a return to see the move it needed.