Convex
LibraryA backend with queries and mutations. The official default rank is no guidelines and no web access.
Apache-2.0 · www.convex.dev · github.com/get-convex/convex
Framework eval
Convex Evals · Framework eval · 2026-10-08 · 92a60e0bb2f156b5ac44e6ea3f5d03a91deb434b9d002d14819d1fc2b6e35cdb
Convex writes the tasks and grades them. This measures a model together with Convex, not one framework against another. The front is near the ceiling. With Guidelines is omitted: on 2026-10-10 the public query runs:leaderboardScores with experiment=with_guidelines returned a server error. The With Guidelines tab has no experiment field. The unscoped default query is not treated as that arm.
| # | Model | Baseline | No guidelines + web | Cost per run |
|---|---|---|---|---|
| 1 | Claude Opus 5.5 1 run | 90.18 | 96.43 | $3.64 |
| 2 | GPT-6.1 Sol 1 run | 88.39 | 91.96 | $1.21 |
| 3 | Claude Sonnet 5.5 1 run | 84.82 | 84.82 | $1.42 |
| 4 | Claude Opus 5 1 run | 80.36 | 83.93 | $4.53 |
| 5 | Grok 4.7 1 run | 80.36 | — | $4.65 |
| 6 | Claude Haiku 5.5 2 runs | 74.55 | 79.02 | $0.16 |
| 7 | Gemini 3.1 Pro Preview 1 run | 72.32 | — | $3.99 |
| 8 | GPT 6 Luna 1 run | 67.86 | 67.86 | $0.08 |
| 9 | DeepSeek V4.1 Flash 1 run | 66.07 | 75.0 | $0.16 |
| 10 | Gemini 3.8 Flash 1 run | 64.29 | — | $2.40 |
| 11 | MiMo V2.6 Pro 1 run | 54.46 | — | $0.40 |
| 12 | MiMo V2.6 Flash 1 run | 48.21 | — | $0.10 |
| 13 | GLM-5.3 1 run | 42.86 | 39.29 | $0.45 |
Install and use
npm i convex
Why this, and when not
Why The 112-task no-guidelines board stands on its own. The web arm is another column and is not merged into the rank.
Skip it If you already use Supabase or your own database, do not switch to add two official boards together.