The Known Good — AI model & API provider analysis

Quality

Quality of leading AI models, aggregated from published independent evaluations

Known Good IndexCoding IndexAgentic Index

Known Good Index

Known Good Index v1.2 incorporates 6 evaluations: GPQA Diamond, MATH / AIME, SWE-bench Verified, LiveBench, Humanity's Last Exam, Terminal-Bench. Normalised 0–100. Scores ingested from Epoch AI (CC-BY) and LiveBench.

22 of 61 ranked ▾ ☰⚙⇩ +Add model from specific provider

98

96

95

94

93

91

90

88

83

82

80

78

77

74

73

72

gpt-5.5-pre-release

💡

Gemini 3.1 Pro
Preview

💡

💡

💡

💡

💡

💡

💡

💡

💡

💡

💡

Gemini 3 Flash
Preview

💡

💡

💡

💡

💡

💡

💡

claude-opus-4-5-20251101

💡 Reasoning models are indicated by a lightbulb

The Known Good

Known Good Index v1.2 combines six public benchmarks, each min-max normalised across tracked models then weighted equally. We do not run these evaluations ourselves — scores are ingested from Epoch AI (CC-BY) and LiveBench. See the Index methodology for weights, normalisation, and per-eval provenance.

Coding Index

Software engineering and agentic terminal ability

Coding IndexSWE-bench VerifiedTerminal-BenchLiveBench CodingAgentic Index

Coding Index

Coding Index, normalised 0–100 from the published evaluations that feed it. Ingested from Epoch AI (CC-BY) and LiveBench. We do not run these evaluations.

19 of 19 ranked ▾ ☰⚙⇩ +Add model from specific provider

100

89

85

78

77

76

74

71

68

67

64

61

60

50

42

27

💡

💡

💡

💡

claude-opus-4-5-20251101

💡

Gemini 3 Flash
Preview

💡

GPT-5.2

gemini-3-pro-preview

claude-3-7-sonnet-20250219

💡

💡

💡

💡

💡

claude-sonnet-4-5-20250929

claude-opus-4-1-20250805

💡

GPT-5 Mini

💡

Gemini 2.5 Pro

GPT-4o

💡 Reasoning models are indicated by a lightbulb

The Known Good

Image & Video Leaderboards

Elo from blind human preference votes, with 95% confidence intervals

Text to ImageImage EditingText to VideoImage to Video

Text to Image Arena

Elo rating from blind pairwise preference votes. Ingested from Arena, updated daily. Higher is better.

12 of 79 models ▾ ☰⚙⇩ +Add model from specific provider

The Known Good

Speech Leaderboards

Text-to-speech quality from blind listening preference votes

Text to Speech Elo

Text to Speech Arena

Elo rating from blind A/B listening tests. Ingested from TTS Arena, updated daily. Higher is better.

12 of 42 models ▾ ☰⚙⇩ +Add model from specific provider

1565

1562

1560

1551

1550

1544

1542

1536

1534

1532

1531

MiniMax Speech 2.8
HD

The Known Good

Capability Indices

Capability-specific indices built from the relevant evaluations

MathematicsScienceReasoningAgentic

Mathematics Index

Mathematics Index, normalised 0–100 from the published evaluations that feed it. Ingested from Epoch AI (CC-BY) and LiveBench. We do not run these evaluations.

22 of 166 ranked ▾ ☰⚙⇩ +Add model from specific provider

100

99

98

97

96

95

gpt-5.5-pro-pre-release

💡

💡

💡

💡

💡

💡

💡

💡

💡

Gemini 2.5 Pro
Preview 05-06

gemini-2.5-pro-preview-03-25

💡

💡

💡

💡

💡

💡

💡

Gemini 3.1 Pro
Preview

💡

Gemini 3.5 Flash

💡

GPT-5.4

💡

Qwen3.7 Max

💡 Reasoning models are indicated by a lightbulb

The Known Good

Quality Breakdown

Individual evaluations behind the index, as measured by their publishers

GPQA DiamondMATH / AIMESWE-bench VerifiedLiveBenchHumanity's Last ExamTerminal-Bench

GPQA Diamond

Graduate-level science questions written to be Google-proof. Aggregated from Epoch AI's AI Benchmarking Hub (gpqa_diamond.csv, column 'mean_score'), used under CC-BY 4.0. The Known Good aggregates published results; it runs no evaluations. Higher is better.

16 of 908 models ▾ ☰⚙⇩ +Add model from specific provider

95

94

93

92

91

💡

GPT-5.4 Pro

💡

Gemini 3.1 Pro
Preview

gpt-5.5-pre-release

gpt-5.5-pro-pre-release

💡

💡

💡

💡

💡

💡

💡

💡

💡

💡

💡

💡 Reasoning models are indicated by a lightbulb

The Known Good

Arena Elo

Human preference rating from blind pairwise votes

OverallStyle controlled

Text Arena

Elo rating from blind pairwise preference votes. Ingested from Arena, updated daily. Higher is better.

12 of 381 models ▾ ☰⚙⇩ +Add model from specific provider

1503

1497

1493

1491

1490

1489

1485

1483

1482

1480

1479

1475

The Known Good

Openness Index

Weight availability and licence permissiveness

Openness Index

Openness Index — distribution

How many tracked models sit in each openness band. Scores weight availability, licence permissiveness, and commercial-use restrictions, derived from models.dev and Epoch AI metadata, not from evaluations. Bars are bands, not models, so bar colour carries no creator here.

908 of 908 models ▾ ☰⚙⇩ +Add model from specific provider

Each bar is an openness band; the value is how many tracked models sit in itBands: 100 open weights, permissive · 90 open weights, permissive, no retrievable repository · 60 open weights, restricted use · 40 open weights, non-commercial · 5 proprietary

74

11

110

7

706

Open weights,
permissive

Open weights,
no repository

Open weights,
restricted use

Open weights,
non-commercial

Proprietary

The Known Good

Context Window

Maximum input tokens accepted

Context windowMax output tokens

Context Window

Maximum input context, from provider metadata via models.dev and OpenRouter. Higher is better.

12 of 908 models ▾ ☰⚙⇩ +Add model from specific provider

2.00M

1.31M

1.05M

Grok 4.20

Auto Router (Beta)

Pareto Code Router

Grok 4.20
Multi-Agent

The Known Good

Output Tokens

Maximum tokens a model will emit in a single response

Max output tokensContext window

Max Output Tokens

Provider-declared maximum completion length, from provider metadata via models.dev and OpenRouter. Higher allows longer single-pass generation.

12 of 908 models ▾ ☰⚙⇩ +Add model from specific provider

2.00M

1.00M

512k

500k

384k

262k

The Known Good

Price and Cost

Published API pricing, blended and by token type

Blended priceInput priceOutput priceCache hit priceQuality per dollar

Blended Price

USD per 1M tokens at a 3:1 input:output blend. Live from OpenRouter, cross-checked against models.dev. Lower is better. Excludes 18 models published at no cost (free and promotional tiers).

13 of 908 models ▾ ☰⚙⇩ +Add model from specific provider

$0.01

$0.02

$0.04

$0.06

Llama 3.1 8B
Instruct

The Known Good

We publish blended price rather than cost-per-task. Cost-per-task requires token counts from a proprietary evaluation suite we do not run. Blended price is computed from published rates and is fully reproducible — see pricing methodology.

Speed & Latency

Throughput and time-to-first-token, measured across hosting providers

Output speedLatency (TTFT)

Output Speed

Median output tokens per second across all tracked providers. Derived from OpenRouter endpoint telemetry. Higher is better.

12 of 908 models ▾ ☰⚙⇩ +Add model from specific provider

The Known Good

API Provider Performance

How inference hosts compare when serving the same model

Output speedLatencyPriceContext served

Output Speed by Provider: gpt-oss-120b

Median tokens per second per host, serving the same model (gpt-oss-120b). Derived from OpenRouter endpoint telemetry. Higher is better.

10 of 17 providers ▾ ☰⚙⇩ +Add model from specific provider

495

411

306

285

281

246

207

181

166

111

The Known Good

Trends

How frontier model quality has moved over time, by creator

Quality over time

Frontier Model Quality, Over Time

Known Good Index v1.2 incorporates 6 evaluations: GPQA Diamond, MATH / AIME, SWE-bench Verified, LiveBench, Humanity's Last Exam, Terminal-Bench. Each step is a new release that raised that creator's best score. Scores ingested from Epoch AI (CC-BY) and LiveBench.

7 of 12 model creators ▾ ☰⚙⇩ +Add model from specific provider

OpenAIGoogleAnthropicAlibabaDeepSeekZ.aixAI

The Known Good

Reference data for AI models

Quality

Coding Index

Image & Video Leaderboards

Speech Leaderboards

Capability Indices

Quality Breakdown

Arena Elo

Openness Index

Context Window

Output Tokens

Price and Cost

Speed & Latency

API Provider Performance

Trends

Hugging Face Wasn't the Target. It Was in the Way.

Delaware County's Cyberattack and the Missing 911 Boundary

When the Router Exfiltrates Its Own Configuration

AI Worked Both Sides of the Ledger This Week. The Lesson Isn't "Patch Faster."

Pennsylvania Says Its Statewide 911 Disruption Wasn't a Cyberattack. The Alternative May Be Harder to Defend Against.

PamStealer Skips the Process Chains Defenders Watch. Not All of Them.

Your Agent Framework Is a Pile of API Keys on a Public IP

There's No Patch for FortiBleed. Public-Sector Networks Are Where That Hurts Most.

Four Failures, One Excuse: "Patch Faster" Was Never the Answer

Vibe Coding Isn't the Problem. Not Understanding the Stack Is.

I Handed Claude Code the Keys. Turns Out I'm Not the Only One Using Them.

The Circuit Nobody Could Find: How Exact-Match Searching Nearly Cost Me the Audit

The HTTP/2 Bomb Sat in Plain Sight for a Decade. An AI Just Had to Read the Code.

The Copilot Meter Didn't Raise the Price. It Showed You the Bill.

Designing AI for a Teen Discord Server Without Turning It Into a Surveillance Machine

WhatsApp Says No One Can Read Your Messages. A Federal Agent Spent 10 Months Disagreeing.

The Patch Queue Is the New Vulnerability

The Week the Toolchain Became the Kill Chain

ShinyHunters Didn't Breach 9,000 Schools. They Breached One Vendor. Your Institution Inherited the Rest.

The Floor Has Been Hit: Navigating the 2026 Systems Engineering Realignment