Weekly scorecards › 2026-46
Install scorecard: week 2026-46
From 2026-11-12 to 2026-11-14 (3 nightly runs), 910 of 1045 judgeable installs passed on the last night (87.1%). 2 confirmed broke, 41 really fixed, and 13 flip-flopped, which we refuse to count as news. The most reliable agent was Claude Code at 91.9%.
Agent reliability ranking
| # | Agent | Pass rate | Installs passed | Flaky | Needs a key |
|---|---|---|---|---|---|
| 1 | Claude Code | 91.9% | 1008/1097 | 6 | 16 |
| 2 | Grok Build | 91.6% | 1046/1142 | 8 | 16 |
| 3 | Codex | 78.3% | 267/341 | 2 | 16 |
| 4 | Pi | 59.4% | 240/404 | 3 | 16 |
| 5 | Cursor | 53.4% | 78/146 | 2 | 16 |
New test baseline from 2026-11-12: the nightly moved to newer releases of all 5 agents that day, so this week's numbers are not like-for-like with earlier weeks. "Confirmed broke" and "really fixed" only compare nights on the same baseline. What changed · No valid run on 2026-11-09, 2026-11-10, 2026-11-11 (test-harness bug, fixed)
Standout of the week
MCP time (reference): clean in 5 agents with zero flakes. Time and timezone conversion
Confirmed broke (2)
- Airwallex Dev (2 agent-nights)
Really fixed (41)
Flaky (13)
Pass one night, fail the next. Not broken, not fixed, just moody.
- @cyanheads/pubmed-mcp-server
- @rigour-labs/mcp
- comfyui-mcp
- @mozilla/firefox-devtools-mcp-moz
- mcp-local-rag
25 installs failed only because they need an API key our sandbox doesn't have. We count those separately.
Built from the nightly daily reports. Check a tool for today · How we test
Get this scorecard by email every Tuesday
Free, every Tuesday: this scorecard plus new skills and what broke. The daily report stays on the site, no daily emails.
Loading the signup form… or subscribe on Beehiiv
Form not loading? Subscribe on Beehiiv →