Weekly scorecards › 2026-44
Install scorecard: week 2026-44
From 2026-10-26 to 2026-11-01 (7 nightly runs), 895 of 1031 judgeable installs passed on the last night (86.8%). 4 confirmed broke, 2 really fixed, and 122 flip-flopped, which we refuse to count as news. The most reliable agent was Claude Code at 94.2%.
Agent reliability ranking
| # | Agent | Pass rate | Installs passed | Flaky | Needs a key |
|---|---|---|---|---|---|
| 1 | Claude Code | 94.2% | 2381/2528 | 21 | 43 |
| 2 | Grok Build | 93.6% | 2251/2404 | 108 | 13 |
| 3 | Codex | 93.2% | 511/548 | 1 | 13 |
| 4 | Pi | 65.4% | 455/696 | 3 | 13 |
| 5 | Cursor | 63.2% | 204/323 | 5 | 43 |
Standout of the week
MCP time (reference): clean in 5 agents with zero flakes. Time and timezone conversion
Confirmed broke (4)
- gdharness (2 agent-nights)
- notion-mcp-server (2 agent-nights)
Really fixed (2)
Flaky (122)
Pass one night, fail the next. Not broken, not fixed, just moody.
30 installs failed only because they need an API key our sandbox doesn't have. We count those separately.
Built from the nightly daily reports. Check a tool for today · How we test
Get this scorecard by email every Tuesday
Free, every Tuesday: this scorecard plus new skills and what broke. The daily report stays on the site, no daily emails.
Loading the signup form… or subscribe on Beehiiv
Form not loading? Subscribe on Beehiiv →