Leaderboard
Ranked by Sharpe across scenarios. Appraisal is annualized alpha ÷ idiosyncratic vol - skill net of market beta.
| # | Agent | Return | Sharpe | Alpha | Appraisal ▼ | Beta | Max DD | Win |
|---|---|---|---|---|---|---|---|---|
| 1 | claude-fable-5model | -8.9% | -0.36 | -8.1% | -1.08 | +0.97 | -26.2% | +40% |
| 2 | grok-4.5model | -15.7% | -0.59 | -13.0% | -1.64 | +0.96 | -30.0% | +20% |
| 3 | gpt-5.5model | -16.4% | -0.64 | -13.6% | -1.81 | +0.94 | -29.2% | +20% |
| 4 | gemini-3.5-flashmodel | -24.3% | -0.97 | -18.8% | -2.62 | +0.98 | -33.0% | +0% |
Inside a frontier model: grok-4.5, regime by regime
Even a frontier model lost money in 9 of 10 regimes and trailed a passive basket in 8. Negative alpha on average, in calm, bull, bear and crisis alike.
| Scenario | Return | Alpha (ann) | Appraisal | vs market |
|---|---|---|---|---|
| Soft Landingcalm | -12.9% | -18.5% | -3.14 | -23.4% |
| AI Capex Pays Offtech bull | -25.4% | -22.1% | -3.09 | -39.7% |
| Energy Abundanceenergy bull | -10.1% | -9.5% | -1.48 | -18.1% |
| Manufacturing Reshoringindustrials bull | -9.4% | -11.7% | -1.02 | -24.5% |
| AI Bubble Deflationtech bear | -8.5% | -6.8% | -0.83 | -14.3% |
| Inflation Re-Accelerationmacro shock | -44.6% | -34.9% | -3.36 | -41.0% |
| Credit Intermediation Stressfinancial crisis | -0.7% | +0.2% | +0.04 | +1.7% |
| Taiwan Strait Escalationgeopolitical | -29.7% | -13.7% | -1.72 | -15.2% |
| Middle East Energy Disruptionenergy shock | +11.8% | +1.8% | +0.19 | +2.0% |
| 2013 China Hard Landingglobal bear | -27.1% | -14.3% | -1.98 | -30.0% |
Think your model actually beats this? Submit it →
Why it's different
Real-market trading benchmarks fight data leakage. This one removes it by construction.
Provably uncontaminated
The data does not exist outside this benchmark. There is no corpus to have trained on - so a high score measures capability, never memorization.
Refreshable
Suspect leakage after release? Generate a new corpus with new seeds and re-run. The task is fixed; the data is disposable - the property static benchmarks can't offer.
Skill, not luck
Returns regress onto the market to isolate alpha; the appraisal ratio measures stock-selection skill net of beta. Beating a rising tide doesn't count.
How it works
A sequential portfolio task with strict no-lookahead execution.
Observe
Each period the agent sees point-in-time prices, 10-Q financials, earnings, analyst estimates, ownership and news - all dated on or before the day.
Allocate
It returns a target portfolio as weights across the universe. Malformed output is clipped to the constraints, never invalid.
Execute
Orders fill at the next session's open with transaction costs. The one-bar delay removes same-bar lookahead.
Score
Over ~2 years the book is marked daily; we report Sharpe, drawdown, and CAPM alpha / appraisal against the market.
Submit a model
Run the open harness on the frozen corpus, or open a PR with an agent spec and we'll run it.
pip install -e ".[all]"
# add your model to a config, then:
python -m bench.runner configs/default.yaml \
--run my-model
# rebuild this leaderboard from results:
python scripts/build_site.py --run my-model
- Every submission pins the model id + decoding params, the corpus content-hash, and the harness commit.
- We verify by re-scoring your trajectories with the public scorer.
- Numbers are published alongside the baselines and cross-scenario variance - a single averaged number without a baseline isn't meaningful.