Today's edition

Civic Broadsheet · Data desk
GovernBenchModels in the chamber. One ledger.

Costs

What a run will cost

An estimate from the treasury, not an invoice.

v2v3survival

Mean of each model across 54 finished runs in v3. One dot is a model, not a single sitting. Survival stays off this mean.

Next: read the line items, or go back to results.

Saved answers

A reused call was answered from cache. Unmarked calls have no cache status.

Calls logged
4458
Reused
6
Fresh
3898
Unmarked
554
Reuse rate
0.1%

Line items

Logged mean per finished run. 54 v3 jobs in total, about $5.70.

  • Claude Haiku 5.5$0.0284 / run
  • GLM 5.3 Flash$0.3172 / run
  • DeepSeek V4.1 Flash 1017$0.1928 / run
  • MiniMax M3.1 Flash$0.0756 / run
  • GPT OSS 20B$0.1633 / run
  • Gemma 4 26B A4B$0.0784 / run
  • Qwen3 30B A3B$0.0283 / run
  • Llama 3.1 8B$0.0192 / run
  • GLM 4.7 Flash$0.0472 / run

Price and performance

Mean of each model across 54 finished runs in v3. One dot is a model, not a single sitting. Survival stays off this mean.

Is a pricier model worth it?

Governance score against average cost of one run, in US dollars (log scale).

Best is top left: higher score, lower cost

0.000.200.400.600.801.00$0.01$0.02$0.05$0.1$0.2$0.5Cost per run (US dollars, log scale)Governance scoreMiniMax M3.1 Flash: score 0.90, $0.076 per run, 6 runsGemma 4 26B A4B: score 0.89, $0.078 per run, 6 runsDeepSeek V4.1 Flash 1017: score 0.89, $0.193 per run, 6 runsClaude Haiku 5.5: score 0.87, $0.028 per run, 6 runsGLM 5.3 Flash: score 0.85, $0.317 per run, 6 runsQwen3 30B A3B: score 0.64, $0.028 per run, 6 runsGLM 4.7 Flash: score 0.64, $0.047 per run, 6 runsGPT OSS 20B: score 0.63, $0.163 per run, 6 runsLlama 3.1 8B: score 0.44, $0.019 per run, 6 runs
  • MiniMax M3.1 Flash: score 0.90, $0.076 per run, 6 runs
  • Gemma 4 26B A4B: score 0.89, $0.078 per run, 6 runs
  • DeepSeek V4.1 Flash 1017: score 0.89, $0.193 per run, 6 runs
  • Claude Haiku 5.5: score 0.87, $0.028 per run, 6 runs
  • GLM 5.3 Flash: score 0.85, $0.317 per run, 6 runs
  • Qwen3 30B A3B: score 0.64, $0.028 per run, 6 runs
  • GLM 4.7 Flash: score 0.64, $0.047 per run, 6 runs
  • GPT OSS 20B: score 0.63, $0.163 per run, 6 runs
  • Llama 3.1 8B: score 0.44, $0.019 per run, 6 runs

Dashed line: the best score on offer at each price

Claude Haiku 5.5 scores 0.87, 0.03 below MiniMax M3.1 Flash, at $0.028 per run. MiniMax M3.1 Flash costs 2.7x as much.

One run is one full game of governing through a series of crises. Cost is what the model's tokens charge for that game.

What does one run cost?

Average cost of one run per model, in US dollars.

Lower is better

Llama 3.1 8B6 runs
$0.019
Qwen3 30B A3B6 runs
$0.028
Claude Haiku 5.56 runs
$0.028
GLM 4.7 Flash6 runs
$0.047
MiniMax M3.1 Flash6 runs
$0.076
Gemma 4 26B A4B6 runs
$0.078
GPT OSS 20B6 runs
$0.163
DeepSeek V4.1 Flash 10176 runs
$0.193
GLM 5.3 Flash6 runs
$0.317

Cost per run (US dollars)

Llama 3.1 8B is cheapest at $0.019. GLM 5.3 Flash costs 16x as much.

A run is one full game of governing through a series of crises.

How much does each model read and write?

Average tokens used in one run, input and output combined.

Lower is better

Claude Haiku 5.56 runs
112k
MiniMax M3.1 Flash6 runs
230k
Llama 3.1 8B6 runs
288k
Qwen3 30B A3B6 runs
335k
GLM 4.7 Flash6 runs
366k
Gemma 4 26B A4B6 runs
604k
DeepSeek V4.1 Flash 10176 runs
678k
GPT OSS 20B6 runs
778k
GLM 5.3 Flash6 runs
1463k

Tokens per run

GLM 5.3 Flash reads and writes the most, 1463k tokens, 13x the least (Claude Haiku 5.5).

A token is a small chunk of text, roughly three quarters of a word. Fewer tokens means a cheaper, faster run.