Skip to content
Back to model rankings

How ranks are calculated

Combine public benchmarks to understand the evidence and method behind the ranking.

0—100

Shared evidence, a clear order.

Combines public benchmarks and minimizes conflict with known results across the full ranking. The overall board and four category boards each show up to the top 30.

The consensus index converts evidence supporting the original ranking into 0–100 for easy comparison. It is not an accuracy rate or a percentage gap in capability. Close support can produce tied scores; rank is still decided by the full evidence.

  1. 01

    First, confirm it's the same model

    We unify names and versions across benchmarks. For different reasoning tiers of one model, a single representative config is chosen by preset rules, not by highest score. Anonymous test codenames and results that mixed in other models are excluded; officially released Preview versions may take part.

  2. 02

    Compare only models that were actually tested

    Only real results from the same benchmark are compared. When both sides have published errors, tiny differences within the margin are treated closer to a tie, not a decisive win. If a model wasn't tested, that comparison doesn't exist.

  3. 03

    Set the voting weight, then combine shared opinions

    Each benchmark uses a preset budget. The overall ranking needs at least three benchmark organizations, three evidence families and three specialty areas. Repeat fetches or display slices of one source add no voting weight.

  4. 04

    Find the complete ranking with the fewest conflicts

    Benchmarks can disagree. We choose the order that violates the least shared net vote weight, then calculate the display index separately. Missing benchmarks are never scored as zero, and unpublished results are never guessed.

A BALANCED VIEW

Composite tests, blind human votes and specialty benchmarks, read together.

Composite benchmarks take 30%, blind human votes 10%, and specialty benchmarks 60% combined. New benchmarks are first checked for overlap in public documentation, then allotted from the matching share; missing shares aren't passed to other benchmarks.

  • 综合评测30%AA Index
  • 真人盲选10%Arena Text · Arena 创作盲选
  • 编程与设计12%Arena WebDev · DeepSWE v1.1 · TapTap Maker · LiveBench · 编程综合 · CursorBench · ALE-Bench
  • 写作与表达9%LiveBench · 语言与指令 · Creative Writing v3 · Longform Writing
  • 数学与推理12%LiveBench · 推理与数学 · FrontierMath v2 · Tiers 1–3 · FrontierMath v2 · Tier 4 · Chess Puzzles · Mystery Game Puzzles · OTIS Mock AIME
  • 知识与事实6%SimpleQA Verified · GPQA Diamond
  • 视觉理解6%Arena Vision
  • 工具与办公6%APEX-Agents 1.1 · Vending-Bench 2 · τ³-Banking
  • 中文与多语言6%AA 中文／多语言
  • 行业专业任务3%Vals Finance Agent

Coding, reasoning, knowledge and professional work each draw on real benchmarks, with category scores calculated independently. Vision and multilingual evidence stay in the overall ranking; renaming categories adds no voting weight to one result. The overall score isn't the mean of category scores. A category usually needs at least two valid benchmarks and five comparable models to show. Knowledge currently relies on two Epoch benchmark sets, with the same-organization limitation stated openly. Creative preference and web development evidence still feed the overall ranking; the first version has no separate aesthetics or writing board.

You may also want to know

Why can a model with little evidence make the list?
The number of benchmarks and capability are different things. The overall ranking needs at least three organizations and cross-capability evidence. Most categories need two organizations; knowledge uses two benchmarks from one organization, stated openly. Missing benchmarks score no zero, but can still cause bias.
What does “evidence-sensitive” mean?
A model is marked evidence-sensitive when removing one benchmark or organization, shifting a weight by 20%, or changing error handling moves its rank range by three or more, costs it eligibility in some scenarios, or leaves comparisons incomplete. The scenario range on the detail page is not a 95% confidence interval and excludes unknown results.
Does a higher model always win head-to-head?
Not necessarily. A may beat B, B beat C, and C beat A. A full ranking must weigh these conflicts, so non-adjacent ranks can differ from a head-to-head comparison. Mathematically optimal means the fewest total conflicts under the current rules; it does not prove the order of capability across all real work.
Why can rankings differ from my experience?
The rankings combine public benchmarks and reflect the overall capability those results support. Real experience also depends on product version, reasoning tier, tool environment and long-task stability. New benchmarks test old rankings; when scores are close, a difference of one or two places shouldn't be read as a clear gap.
When do new models appear?
AIDC热点 checks upstream results four times a day. Only new results actually published by a benchmark count toward ranking; page refreshes, price updates or our fetch times are not re-evaluations.
What if a benchmark is temporarily unavailable?
Still-valid source snapshots stay in use. Within the same benchmark version, rows that temporarily drop out carry over the latest verified record for up to seven days. Valid results kept on a public board are not removed just because they stay unchanged. Official withdrawals, corrections and new versions invalidate or replace records; historical highs are not kept. If a whole round fails, the previous valid ranking and its time remain.
Do composite indexes double count specialty benchmarks?
We check overlap against public task sets and method notes, and cap the combined share of related sources; unconfirmed correlation remains a limitation. The AA index currently holds 30%. Arena's overall text and creative specialty share the original 10% human-preference share, 5% each. They have overlapping votes, so they still count as one evidence family rather than two independent benchmarks.
Do price or speed affect rank?
Neither. Price only helps you estimate API costs. It's shown per million tokens with the official vendor source kept, and doesn't represent subscription fees.
See calculation details and the current version

Method version:2026.10-public-consensus-v16. Uses weighted incomplete Kemeny ranking. Net support M is summed for each pair of co-evaluated models; the goal is to minimize the total net support ranked backwards. Integer optimization returns optimality status and objective bounds, and only fully validated results are published.

When both sides report a standard error, net support is 2Φ(score gap / combined standard error) − 1, assuming zero covariance by default; other comparisons take only the direction of the raw lead. Unknown error isn't zero error, and treating small gaps ordinally remains a limitation. Voting weight applies to potential model pairs, so sources covering more models use more comparison slots; nominal budgets shouldn't be read as precise contributions to the final ranking.

The display index keeps the original order: it swaps each pair of adjacent models, lets the other models reorder, and computes the smallest added reverse net support. These non-negative differences are summed along the original ranking, then mapped to 0–100 with a sigmoid relative to the fixed reference group. When an alternative order is equally optimal, the gap stays zero; scores show one decimal and no minimum gap is imposed. The index isn't fed back into ordering. Entry set, reference models and evidence changes still affect it, and gaps don't equal real capability distance. Ties are solved in fixed model ID order. Integer optimization uses the HiGHS solver (highs 1.15.3), and the normal distribution function for error conversion uses the same algorithm as SciPy norm.cdf. Sources, licenses, eligibility and each round's inputs are versioned. No falsely precise order across components is published when they aren't connected by shared evidence.

Fixed reference models:claude-fable-5、gpt-5-6-sol、kimi-k-3、qwen-3-8-max、gpt-5-4、claude-opus-4-8、claude-sonnet-5、grok-4-5、gemini-3-5-flash、glm-5-2、claude-sonnet-4-6、qwen-3-7-max、kimi-k-2-6、deepseek-v-4-pro、qwen-3-6-plus、minimax-m-3、gpt-5-4-mini、grok-4-3. The reference group sets the index scale and does not dictate any vendor's rank. Indexes of different categories are not directly comparable.

See every piece of benchmark evidence →