RLTasksby ulam.ai
Model launches / Kimi

Kimi K3.1 Pro

Scores at a glance

58 benchmarks · large dot: Kimi K3.1 Pro; small dots: closest models where reported

AA-Briefcase v1.1 ↗1677.79 Elo
833.061811.82

Kimi K3.1 Pro · AA-Briefcase v1.1 · overall Elo ↗

1 more reported result

1678 Elo · Kimi K3.1 Pro (max) · AA-Briefcase v1.1 · AA-Briefcase v1.1 (Elo-score) · Kimi K3.1 Pro · complete release comparison · Introducing Kimi K3.1 Pro ↗

AutomationBench-AA ↗73.30%
4.9877.51

Kimi K3.1 Pro · AutomationBench-AA · guardrail-gated objective score ↗

1 more reported result

73.3% · Kimi K3.1 Pro (max) · AutomationBench-AA · Automation Bench-AA · Kimi K3.1 Pro · complete release comparison · Introducing Kimi K3.1 Pro ↗

GDPval-AA v2.1 ↗1845.72 Elo
1071.981866.27

Kimi K3.1 Pro · GDPval-AA v2.1 · Elo ↗

1 more reported result

1846 Elo · Kimi K3.1 Pro (max) · GDPval-AA v2.1 · GDPval-AA v2.1 (Elo-score) · Kimi K3.1 Pro · complete release comparison · Introducing Kimi K3.1 Pro ↗

AA-3D ↗1110.16 Elo
572.551198.83

Kimi K3.1 Pro · AA-3D · Elo ↗

SciCode ↗60.76%
42.9466.9

Kimi K3.1 Pro · SciCode · AA implementation ↗

AA-PersonalityBench ↗62.88 percentile
28.92100

Kimi K3.1 Pro · AA-PersonalityBench · expressiveness ↗

7 more reported results

87.62 percentile · Kimi K3.1 Pro · AA-PersonalityBench · organization · Artificial Analysis ↗

23.28 percentile · Kimi K3.1 Pro · AA-PersonalityBench · informativeness · Artificial Analysis ↗

87.62 percentile · Kimi K3.1 Pro · AA-PersonalityBench · transparency · Artificial Analysis ↗

50.50 percentile · Kimi K3.1 Pro · AA-PersonalityBench · warmth · Artificial Analysis ↗

62.88 percentile · Kimi K3.1 Pro · AA-PersonalityBench · conviction · Artificial Analysis ↗

52.98 percentile · Kimi K3.1 Pro · AA-PersonalityBench · imagination · Artificial Analysis ↗

20.80 percentile · Kimi K3.1 Pro · AA-PersonalityBench · slop · Artificial Analysis ↗

SWE-Marathon ↗51.7%
4051.7

Kimi K3.1 Pro (max) · SWE-Marathon · SWE Marathon v1.1 · Kimi K3.1 Pro · complete release comparison ↗

BenchCAD ↗72.8%
41.273

Kimi K3.1 Pro (max) · BenchCAD · BenchCAD w/o tools · Kimi K3.1 Pro · complete release comparison ↗

1 more reported result

92.7% · Kimi K3.1 Pro (max) · BenchCAD · BenchCAD w/ tools · Kimi K3.1 Pro · complete release comparison · Introducing Kimi K3.1 Pro ↗

ProgramBench ↗31.5%
1965

Kimi K3.1 Pro (max) · ProgramBench · Program Bench · Kimi K3.1 Pro · complete release comparison ↗

DeepSWE v1.1 ↗75.5%
66.977

Kimi K3.1 Pro (max) · DeepSWE v1.1 · Kimi K3.1 Pro · complete release comparison ↗

PostTrainBench ↗36.5%
30.544.6

Kimi K3.1 Pro (max) · PostTrainBench · PostTrain Bench v1.2 · Kimi K3.1 Pro · complete release comparison ↗

Kimi Code Bench 2.0 ↗77.1%
61.777.1

Kimi K3.1 Pro (max) · Kimi Code Bench 2.0 · Kimi Code Bench 2.0 (Internal) · Kimi K3.1 Pro · complete release comparison ↗

Kimi Code Bench 3.0 ↗61.2%
45.566.5

Kimi K3.1 Pro (max) · Kimi Code Bench 3.0 · Kimi Code Bench 3.0 Long-horizon (Internal) · Kimi K3.1 Pro · complete release comparison ↗

BrowseComp ↗92.1%
90.892.1

Kimi K3.1 Pro (max) · BrowseComp · Kimi K3.1 Pro · complete release comparison ↗

BioMysteryBench ↗80.6%
78.981.9

Kimi K3.1 Pro (max) · BioMysteryBench · Kimi K3.1 Pro · complete release comparison ↗

CharXiv ↗93.4%
85.693.8

Kimi K3.1 Pro (max) · CharXiv · CharXiv (RQ) w/ python · Kimi K3.1 Pro · complete release comparison ↗

1 more reported result

89% · Kimi K3.1 Pro (max) · CharXiv · CharXiv (RQ) · Kimi K3.1 Pro · complete release comparison · Introducing Kimi K3.1 Pro ↗

Chartography ↗90.2%
8391

Kimi K3.1 Pro (max) · Chartography · ChartoGraphy w/ python · Kimi K3.1 Pro · complete release comparison ↗

BabyVision ↗95.4%
92.597.7

Kimi K3.1 Pro (max) · BabyVision · BabyVision w/ python · Kimi K3.1 Pro · complete release comparison ↗

ZeroBench ↗63%
5163

Kimi K3.1 Pro (max) · ZeroBench · ZeroBench_main w/ python (pass@5) · Kimi K3.1 Pro · complete release comparison ↗

2 more reported results

29.8% · Kimi K3.1 Pro (max) · ZeroBench · ZeroBench_main · avg@5 · Kimi K3.1 Pro · complete release comparison · Introducing Kimi K3.1 Pro ↗

39% · Kimi K3.1 Pro (max) · ZeroBench · ZeroBench_main · pass@5 · Kimi K3.1 Pro · complete release comparison · Introducing Kimi K3.1 Pro ↗

Agents’ Last Exam ↗43.2%
26.243.2

Kimi K3.1 Pro (max) · Agents’ Last Exam · Agents' Last Exam (ALL) · Kimi K3.1 Pro · complete release comparison ↗

1 more reported result

44.2% · Kimi K3.1 Pro (max) · Agents’ Last Exam · Agents' Last Exam (CLI) · Kimi K3.1 Pro · complete release comparison · Introducing Kimi K3.1 Pro ↗

Vals Finance Agent v2 ↗58.6%
4560.8

Kimi K3.1 Pro (max) · Vals Finance Agent v2 · Frontier Finance · Kimi K3.1 Pro · complete release comparison ↗

HANDBOOK.md ↗44.6%
20.844.6

Kimi K3.1 Pro (max) · HANDBOOK.md · Kimi K3.1 Pro · complete release comparison ↗

OSWorld 2.1 ↗79.6%
69.381.8

Kimi K3.1 Pro (max) · OSWorld 2.1 · OSWorld V2.1 · Kimi K3.1 Pro · complete release comparison ↗

DECK-Arena ↗1456 Elo
11191456

Kimi K3.1 Pro (max) · DECK-Arena · DECK-Arena (Internal, Elo-score) · Kimi K3.1 Pro · complete release comparison ↗

Design Bench ↗957 Elo
7181076

Kimi K3.1 Pro (max) · Design Bench · Design Bench (Internal, Elo-score) · Kimi K3.1 Pro · complete release comparison ↗

Kimi Hallucination Benchmark ↗8.3%
5.523.3

Kimi K3.1 Pro (max) · Kimi Hallucination Benchmark · Kimi Hallu Bench Hard w/ tools (Internal, lower is better) · Kimi K3.1 Pro · complete release comparison ↗

1 more reported result

5.4% · Kimi K3.1 Pro (max) · Kimi Hallucination Benchmark · Kimi Hallu Bench w/o tools (Internal, lower is better) · Kimi K3.1 Pro · complete release comparison · Introducing Kimi K3.1 Pro ↗

GPQA Diamond ↗94.2%
91.796.1

Kimi K3.1 Pro (max) · GPQA Diamond · GPQA-Diamond · Kimi K3.1 Pro · complete release comparison ↗

ExploitBench ↗74.7%
54.4100

Kimi K3.1 Pro (max) · ExploitBench · Kimi K3.1 Pro · complete release comparison ↗

ExploitGym ↗20.7%
1542.4

Kimi K3.1 Pro (max) · ExploitGym · Kimi K3.1 Pro · complete release comparison ↗

Closest models by score

Similar positions on shared benchmark leaderboards. Smaller gaps are closer.

14.6 percentile-point gap

18 of 30 shared benchmark comparisons

20.4 percentile-point gap

29 of 30 shared benchmark comparisons

How similarity is measured

Mean absolute gap in percentile positions on shared benchmark settings; lower is closer. At least 60% overlap is required, with a minimum of three when available. Independent evaluations are used when there are at least three; otherwise published report comparisons are used. Repeated model observations within one setting are averaged before ranking. Writing-style traits and composite indices are excluded. This describes similarity, not an overall quality ranking.

About the model report

Harnesses, task subsets, judges, hardware, and scoring transformations vary by benchmark. The launch footnotes are part of each result’s meaning. Reviewed 2026-10-25.

Read the official model report ↗

34 benchmarks in the report

Search this collection →
AGT / BENCHMARK34 reported results

AA-Briefcase v1.1

Agents produce professional work from large collections of business files, with checks for correctness, analysis, and presentation.

Rubric + preference Elo↗
AGT / BENCHMARK38 reported results

GDPval-AA v2.1

Agents complete occupational work and deliver documents, analyses, and other professional artifacts.

Pairwise Elo↗
AGT / BENCHMARK27 reported results

APEX-Agents-AA

Professional-service tasks require sustained coordination across applications, files, and tools.

Task success↗
AGT / BENCHMARK39 reported results

AutomationBench-AA

Agents orchestrate workflows in simulated SaaS systems while respecting operational constraints.

Objectives completed↗
CTX / BENCHMARK103 reported results

GDP.pdf

Professional questions require reasoning over long PDF documents and meeting task-specific requirements.

All-pass rate↗
COD / BENCHMARK115 reported results

Terminal-Bench 4.0

Complex terminal work spans coding, science, infrastructure, and other technical tasks, with executable checks.

Verified task success↗
SCI / BENCHMARK90 reported results

Terminal-Bench-Science 0.1

Research workflows are completed in a terminal and checked with task-specific tests.

Verified task success↗
RSN / BENCHMARK131 reported results

Humanity’s Last Exam

Expert-authored questions stretch academic reasoning and knowledge across many disciplines.

Answer accuracy↗
RSN / BENCHMARK146 reported results

MMMU-Pro

Academic questions require visual understanding as well as language reasoning, with measures against text-only shortcuts.

Multimodal accuracy↗
AGT / BENCHMARK12 reported results

BrowseComp

Difficult information-seeking questions test a browsing agent’s ability to locate and combine evidence.

Answer accuracy↗
COD / BENCHMARK97 reported results

DeepSWE v1.1

Original engineering problems test coding agents on substantial changes in existing repositories.

Verified task success↗
COD / BENCHMARK11 reported results

SWE-Marathon

Extended software projects test whether an agent can keep making useful progress through long engineering sessions.

Long-horizon task score↗
COD / BENCHMARK81 reported results

ProgramBench

Agents reconstruct programs from observed behavior rather than receiving a conventional repository repair issue.

Behavioral reconstruction score↗
RSN / BENCHMARK22 reported results

PostTrainBench

An agent must improve small language models within a constrained training-compute budget.

Held-out post-training performance↗
AGT / BENCHMARK24 reported results

Agents’ Last Exam

Professional software workflows test whether an agent can deliver the requested result in a real application environment.

Professional task success↗
AGT / BENCHMARK13 reported results

OSWorld 2.1

The OSWorld version reported in recent launches measures computer-use agents on extended application workflows.

Partial and binary task score↗
RSN / BENCHMARK17 reported results

Chartography

Specialist charts test visual reasoning grounded in professional practice.

Professional chart accuracy↗
RSN / BENCHMARK26 reported results

ZeroBench

Challenging visual questions probe failures that remain after ordinary image understanding becomes strong.

Visual task accuracy↗
COD / BENCHMARK14 reported results

BenchCAD

Visual-to-code tasks assess whether a model can construct a CAD representation matching a target shape.

Geometric reconstruction overlap↗
COD / BENCHMARK30 reported results

ExploitBench

A controlled security evaluation measures model capability on vulnerability-related software tasks.

Security capability score↗
COD / BENCHMARK10 reported results

ExploitGym

A controlled security benchmark is used in frontier-model capability and safeguard evaluations.

Security task success↗
FIN / BENCHMARK22 reported results

OfficeQA Pro

Find evidence and reason across workplace document collections. Version 1 and version 2 remain separate evaluations within this collection.

Grounded financial-answer accuracy↗
FIN / BENCHMARK96 reported results

Vals Finance Agent v2

Agentic financial research evaluates whether a system can use tools and evidence to complete analyst work.

Financial research task score↗
SCI / BENCHMARK34 reported results

BioMysteryBench

Biological investigations test agentic reasoning through a scientific problem and its supporting evidence.

Scientific investigation score↗
AGT / BENCHMARK5 reported results

HANDBOOK.md

Agents must use operational documentation while working through a tool-based task.

Policy-grounded workflow score↗
AGT / BENCHMARK5 reported results

DECK-Arena

Presentation-generation tasks assess language-model agents producing slides from practical requests.

Presentation quality score↗
AGT / BENCHMARK5 reported results

Design Bench

Language-model workflows produce graphic-design artifacts from direct or reference-based briefs.

Design artifact score↗
AGT / BENCHMARK34 reported results

MCP Atlas

Multi-tool tasks examine how an agent discovers, selects, and coordinates capabilities exposed through MCP.

Tool-workflow task score↗
RSN / BENCHMARK51 reported results

HLE-Diamond

A named challenging subset of Humanity’s Last Exam appears in recent model launches.

Answer accuracy↗
COD / BENCHMARK9 reported results

Kimi Code Bench 2.0

Kimi’s internal coding evaluation tests workflows selected by the model developer.

Internal coding score↗
COD / BENCHMARK6 reported results

Kimi Code Bench 3.0

Long-horizon coding tasks are used in Kimi’s internal model comparisons.

Internal long-horizon coding score↗
KNW / BENCHMARK12 reported results

Kimi Hallucination Benchmark

Challenging practical-work prompts measure unsupported factual claims in Kimi’s internal evaluation.

Claim-level hallucination rate↗
COD / BENCHMARK12 reported results

xdev-Bench-hard

Kimi reports an internal, difficult security-capability evaluation.

Controlled security score↗
KNW / BENCHMARK88 reported results

Harvey Legal Agent Benchmark

A client matter provides documents and a legal-work request; the agent must deliver a reviewable artifact satisfying expert-written requirements.

Expert-rubric all-pass rate↗

COMPARISON CHECKLIST

Match benchmark version, task subset, model effort, harness, tool access, attempt budget, scoring rule, and fallback behavior. A shared benchmark name is the start of a comparison, not proof that two reported scores use the same contract.

Kimi original report ↗