SRE-Bench

In the GPT-6 Astra report, this name refers to software reverse engineering rather than site-reliability incident handling.26 Oct 2026

One row per model and compatible score measure. Mean = equal-weight arithmetic average of imported reports. Distinct configurations reported for the same model in a source table remain separate. Reports can overlap; the range is not a confidence interval. Unbundle to inspect each observation.
		Evidence	What the result shows
GPT-6 Astra →OpenAI	72.44% 2 reports · range 56.87–88%	SRE-Bench[1][2] View 2 reports 88.0% · SRE-Bench [1] 56.87 % · SRE Bench · Accuracy · Cost / Test: $10.24; Input / Output Cost: $10 / $50; Duration: 17m51s [2]	Published observations span 31.13 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. Leads this displayed comparison.
GPT-6.1 Sol →OpenAI	50.76% 1 report	SRE-Bench[2]Cost / Test: $2.69; Input / Output Cost: $2 / $10; Duration: 32m32s	#2 of 35 in this displayed comparison; 21.68 pp below the leader. $2.69 / test32m 32s
Gemini 4 Argon →Google	44.27% 1 report	SRE-Bench[2]Cost / Test: $30.45; Input / Output Cost: $4 / $20; Duration: 1h35m	#3 of 35 in this displayed comparison; 28.16 pp below the leader. $30.45 / test1h 35m
GPT-5.6 Sol →OpenAI	43.22% 2 reports · range 30.53–55.9%	SRE-Bench[1][2] View 2 reports 55.9% · SRE-Bench [1] 30.53 % · SRE Bench · Accuracy · Cost / Test: $42.52; Input / Output Cost: $4 / $20; Duration: 1h28m [2]	Published observations span 25.37 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. #4 of 35 in this displayed comparison; 29.22 pp below the leader.
Claude Opus 5.5 →Anthropic	33.59% 1 report	SRE-Bench[2]Cost / Test: $34.65; Input / Output Cost: $4 / $20; Duration: 1h37m	#5 of 35 in this displayed comparison; 38.84 pp below the leader. $34.65 / test1h 37m
Daybreak Blue →	31.68% 1 report	SRE-Bench[2]Cost / Test: $54.34; Input / Output Cost: $5 / $30; Duration: 2h54m	#6 of 35 in this displayed comparison; 40.76 pp below the leader. $54.34 / test2h 54m
GPT-6 Sol →OpenAI	30.53% 1 report	SRE-Bench[2]Cost / Test: $20.65; Input / Output Cost: $2 / $10; Duration: 1h03m	#7 of 35 in this displayed comparison; 41.91 pp below the leader. $20.65 / test1h 03m
Claude Sonnet 5.5 →Anthropic	30.15% 1 report	SRE-Bench[2]Cost / Test: $26.52; Input / Output Cost: $2 / $10; Duration: 1h55m	#8 of 35 in this displayed comparison; 42.29 pp below the leader. $26.52 / test1h 55m
Claude Fable 5.1 →Anthropic	22.9% 1 report	SRE-Bench[2]Cost / Test: $32.23; Input / Output Cost: $10 / $50; Duration: 2h03m	#9 of 35 in this displayed comparison; 49.54 pp below the leader. $32.23 / test2h 03m
Claude Opus 5.5 (CVP) →Anthropic	21.76% 1 report	SRE-Bench[2]Cost / Test: $33.13; Input / Output Cost: $4 / $20; Duration: 3h22m	#10 of 35 in this displayed comparison; 50.67 pp below the leader. $33.13 / test3h 22m
Claude Opus 5 →Anthropic	12.36% 2 reports · range 12.21–12.5%	SRE-Bench[1][2] View 2 reports 12.5% · SRE-Bench [1] 12.21 % · SRE Bench · Accuracy · Cost / Test: $23.63; Input / Output Cost: $5 / $25; Duration: 1h21m [2]	Published observations span 0.29 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. #11 of 35 in this displayed comparison; 60.08 pp below the leader.
Wonder Full →	11.83% 1 report	SRE-Bench[2]Cost / Test: $33.81; Input / Output Cost: $4 / $20; Duration: 5h34m	#12 of 35 in this displayed comparison; 60.61 pp below the leader. $33.81 / test5h 34m
Gemini 3.8 Flash →Google	11.07% 1 report	SRE-Bench[2]Cost / Test: $27.46; Input / Output Cost: $1.5 / $7.5; Duration: 2h28m	#13 of 35 in this displayed comparison; 61.37 pp below the leader. $27.46 / test2h 28m
GPT-5.6 Terra →OpenAI	9.54% 1 report	SRE-Bench[2]Cost / Test: $30.86; Input / Output Cost: $2 / $12; Duration: 3h17m	#14 of 35 in this displayed comparison; 62.9 pp below the leader. $30.86 / test3h 17m
GLM 5.3 →Z.ai	9.16% 1 report	SRE-Bench[2]Cost / Test: $27.47; Input / Output Cost: $1.4 / $4.4; Duration: 2h48m	#15 of 35 in this displayed comparison; 63.28 pp below the leader. $27.47 / test2h 48m
Claude Haiku 5.5 →Anthropic	5.72% 1 report	SRE-Bench[2]Cost / Test: $3.14; Input / Output Cost: $0.1 / $0.5; Duration: 57m56s	#16 of 35 in this displayed comparison; 66.72 pp below the leader. $3.14 / test57m 56s
Gemini 3.7 Flash →Google	4.58% 1 report	SRE-Bench[2]Cost / Test: $21.47; Input / Output Cost: $1.5 / $7.5; Duration: 1h50m	#17 of 35 in this displayed comparison; 67.86 pp below the leader. $21.47 / test1h 50m
MiMo V2.6 Flash →Xiaomi	4.2% 1 report	SRE-Bench[2]Cost / Test: $0.40; Input / Output Cost: $0.14 / $0.28; Duration: 3h22m	#18 of 35 in this displayed comparison; 68.23 pp below the leader. $0.40 / test3h 22m
GPT-5.5 →OpenAI	3.82% 1 report	SRE-Bench[2]Cost / Test: $17.77; Input / Output Cost: $5 / $30; Duration: 46m28s	#19 of 35 in this displayed comparison; 68.62 pp below the leader. $17.77 / test46m 28s
Grok 4.7 →xAI	3.82% 1 report	SRE-Bench[2]Cost / Test: $64.77; Input / Output Cost: $2 / $6; Duration: 1h41m	#19 of 35 in this displayed comparison; 68.62 pp below the leader. $64.77 / test1h 41m
Muse Spark 1.3 →Meta	3.82% 1 report	SRE-Bench[2]Cost / Test: $25.86; Input / Output Cost: $1.25 / $4.25; Duration: 1h47m	#19 of 35 in this displayed comparison; 68.62 pp below the leader. $25.86 / test1h 47m
MiMo V2.6 Pro →Xiaomi	3.05% 1 report	SRE-Bench[2]Cost / Test: $0.53; Input / Output Cost: $0.435 / $0.87; Duration: 2h23m	#22 of 35 in this displayed comparison; 69.39 pp below the leader. $0.53 / test2h 23m
GPT-6 Luna →OpenAI	2.67% 1 report	SRE-Bench[2]Cost / Test: $1.55; Input / Output Cost: $0.1 / $0.5; Duration: 1h21m	#23 of 35 in this displayed comparison; 69.77 pp below the leader. $1.55 / test1h 21m
Hy4 Preview →	2.29% 1 report	SRE-Bench[2]Cost / Test: $4.90; Input / Output Cost: $0.834 / $2.501; Duration: 1h40m	#24 of 35 in this displayed comparison; 70.14 pp below the leader. $4.90 / test1h 40m
DeepSeek V4 Pro 0813 →DeepSeek	1.91% 1 report	SRE-Bench[2]Cost / Test: $5.58; Input / Output Cost: $1.32 / $3.96; Duration: 2h20m	#25 of 35 in this displayed comparison; 70.53 pp below the leader. $5.58 / test2h 20m
GPT-5.6 Luna →OpenAI	1.53% 1 report	SRE-Bench[2]Cost / Test: $3.44; Input / Output Cost: $0.2 / $1.2; Duration: 1h42m	#26 of 35 in this displayed comparison; 70.91 pp below the leader. $3.44 / test1h 42m
DeepSeek V4.1 Flash →DeepSeek	0.76% 1 report	SRE-Bench[2]Cost / Test: $0.55; Input / Output Cost: $0.3 / $1.2; Duration: 1h01m	#27 of 35 in this displayed comparison; 71.67 pp below the leader. $0.55 / test1h 01m
GLM 5.3 Flash →Z.ai	0.76% 1 report	SRE-Bench[2]Cost / Test: $2.29; Input / Output Cost: $0.075 / $0.25; Duration: 3h15m	#27 of 35 in this displayed comparison; 71.67 pp below the leader. $2.29 / test3h 15m
Grok 4.5 →xAI	0.76% 1 report	SRE-Bench[2]Cost / Test: $13.42; Input / Output Cost: $2 / $6; Duration: 1h06m	#27 of 35 in this displayed comparison; 71.67 pp below the leader. $13.42 / test1h 06m
Claude Sonnet 5 →Anthropic	0.38% 1 report	SRE-Bench[2]Cost / Test: $18.49; Input / Output Cost: $2 / $10; Duration: 3h53m	#30 of 35 in this displayed comparison; 72.06 pp below the leader. $18.49 / test3h 53m
Step 5 Preview →StepFun	0.38% 1 report	SRE-Bench[2]Cost / Test: $3.07; Input / Output Cost: $1 / $2.7; Duration: 1h34m	#30 of 35 in this displayed comparison; 72.06 pp below the leader. $3.07 / test1h 34m
Gemini 3.1 Pro Preview (02/26) →Google	0% 1 report	SRE-Bench[2]Cost / Test: $11.34; Input / Output Cost: $2 / $12; Duration: 1h03m	#32 of 35 in this displayed comparison; 72.44 pp below the leader. $11.34 / test1h 03m
GLM 5.2 →Z.ai	0% 1 report	SRE-Bench[2]Cost / Test: $26.58; Input / Output Cost: $1.4 / $4.4; Duration: 2h29m	#32 of 35 in this displayed comparison; 72.44 pp below the leader. $26.58 / test2h 29m
Inkling →	0% 1 report	SRE-Bench[2]Cost / Test: $2.23; Input / Output Cost: $1 / $4.05; Duration: 28m39s	#32 of 35 in this displayed comparison; 72.44 pp below the leader. $2.23 / test28m 39s
Muse Spark 1.2 →Meta	0% 1 report	SRE-Bench[2]Cost / Test: $23.43; Input / Output Cost: $1.25 / $4.25; Duration: 1h40m	#32 of 35 in this displayed comparison; 72.44 pp below the leader. $23.43 / test1h 40m

Sources

GPT-6 Astra: A new generation of intelligence ↗
OpenAI
Dates & evaluation details
SRE-Bench · 2026-10-26
Launch-reported maxima across reasoning efforts, not a uniformly matched effort comparison. Research/API settings can differ from ChatGPT. Source footnotes disclose altered harnesses, model revisions and reproduced competitor results. A dagger marks cells with additional source qualifications. Preserve each row’s named subset and tools.
SRE Bench — evaluation methodology and results ↗
Vals AI · 2026
Dates & evaluation details
SRE Bench · Accuracy · 2026-10-26
Public source leaderboard captured 26 October 2026. Model names, configurations, native score units and source-reported uncertainty are retained. Version and task split follow this specific Vals methodology page. These are published observations, not evaluations run by RLTasks.

TRACE THE SCORE · Per-model notes use published breakdowns where available.

Score vs. cost per test

Vals AI · 35 observations

SRE Bench · Accuracy · higher is better

Costs are Vals AI’s published Cost / Test in USD, paired with the score from the same result. They reflect that evaluation’s workload and configuration; they are not token prices or cost per successful task. Missing costs are omitted. Explore all reported results ↓

The dashed line traces the best observed score as reported cost increases. It describes this selection of reports, with their original configurations. It does not estimate unmeasured models or costs.

View plotted values and sources (35)

SRE-Bench · SRE Bench · Accuracy. Published measurements in their original score units.
Model	Score	Cost per test (USD)	Configuration & source
GPT-6 Astra →	56.87 %	$10.24	Cost / Test: $10.24; Input / Output Cost: $10 / $50; Duration: 17m51s Vals AI · SRE Bench · Accuracy ↗
GPT-6.1 Sol →	50.76 %	$2.69	Cost / Test: $2.69; Input / Output Cost: $2 / $10; Duration: 32m32s Vals AI · SRE Bench · Accuracy ↗
Gemini 4 Argon →	44.27 %	$30.45	Cost / Test: $30.45; Input / Output Cost: $4 / $20; Duration: 1h35m Vals AI · SRE Bench · Accuracy ↗
Claude Opus 5.5 →	33.59 %	$34.65	Cost / Test: $34.65; Input / Output Cost: $4 / $20; Duration: 1h37m Vals AI · SRE Bench · Accuracy ↗
Daybreak Blue →	31.68 %	$54.34	Cost / Test: $54.34; Input / Output Cost: $5 / $30; Duration: 2h54m Vals AI · SRE Bench · Accuracy ↗
GPT-5.6 Sol →	30.53 %	$42.52	Cost / Test: $42.52; Input / Output Cost: $4 / $20; Duration: 1h28m Vals AI · SRE Bench · Accuracy ↗
GPT-6 Sol →	30.53 %	$20.65	Cost / Test: $20.65; Input / Output Cost: $2 / $10; Duration: 1h03m Vals AI · SRE Bench · Accuracy ↗
Claude Sonnet 5.5 →	30.15 %	$26.52	Cost / Test: $26.52; Input / Output Cost: $2 / $10; Duration: 1h55m Vals AI · SRE Bench · Accuracy ↗
Claude Fable 5.1 →	22.9 %	$32.23	Cost / Test: $32.23; Input / Output Cost: $10 / $50; Duration: 2h03m Vals AI · SRE Bench · Accuracy ↗
Claude Opus 5.5 (CVP) →	21.76 %	$33.13	Cost / Test: $33.13; Input / Output Cost: $4 / $20; Duration: 3h22m Vals AI · SRE Bench · Accuracy ↗
Claude Opus 5 →	12.21 %	$23.63	Cost / Test: $23.63; Input / Output Cost: $5 / $25; Duration: 1h21m Vals AI · SRE Bench · Accuracy ↗
Wonder Full →	11.83 %	$33.81	Cost / Test: $33.81; Input / Output Cost: $4 / $20; Duration: 5h34m Vals AI · SRE Bench · Accuracy ↗
Gemini 3.8 Flash →	11.07 %	$27.46	Cost / Test: $27.46; Input / Output Cost: $1.5 / $7.5; Duration: 2h28m Vals AI · SRE Bench · Accuracy ↗
GPT-5.6 Terra →	9.54 %	$30.86	Cost / Test: $30.86; Input / Output Cost: $2 / $12; Duration: 3h17m Vals AI · SRE Bench · Accuracy ↗
GLM 5.3 →	9.16 %	$27.47	Cost / Test: $27.47; Input / Output Cost: $1.4 / $4.4; Duration: 2h48m Vals AI · SRE Bench · Accuracy ↗
Claude Haiku 5.5 →	5.72 %	$3.14	Cost / Test: $3.14; Input / Output Cost: $0.1 / $0.5; Duration: 57m56s Vals AI · SRE Bench · Accuracy ↗
Gemini 3.7 Flash →	4.58 %	$21.47	Cost / Test: $21.47; Input / Output Cost: $1.5 / $7.5; Duration: 1h50m Vals AI · SRE Bench · Accuracy ↗
MiMo V2.6 Flash →	4.2 %	$0.40	Cost / Test: $0.40; Input / Output Cost: $0.14 / $0.28; Duration: 3h22m Vals AI · SRE Bench · Accuracy ↗
Grok 4.7 →	3.82 %	$64.77	Cost / Test: $64.77; Input / Output Cost: $2 / $6; Duration: 1h41m Vals AI · SRE Bench · Accuracy ↗
GPT 5.5 →	3.82 %	$17.77	Cost / Test: $17.77; Input / Output Cost: $5 / $30; Duration: 46m28s Vals AI · SRE Bench · Accuracy ↗
Muse Spark 1.3 Max →	3.82 %	$25.86	Cost / Test: $25.86; Input / Output Cost: $1.25 / $4.25; Duration: 1h47m Vals AI · SRE Bench · Accuracy ↗
MiMo V2.6 Pro →	3.05 %	$0.53	Cost / Test: $0.53; Input / Output Cost: $0.435 / $0.87; Duration: 2h23m Vals AI · SRE Bench · Accuracy ↗
GPT-6 Luna →	2.67 %	$1.55	Cost / Test: $1.55; Input / Output Cost: $0.1 / $0.5; Duration: 1h21m Vals AI · SRE Bench · Accuracy ↗
Hy4 Preview →	2.29 %	$4.90	Cost / Test: $4.90; Input / Output Cost: $0.834 / $2.501; Duration: 1h40m Vals AI · SRE Bench · Accuracy ↗
DeepSeek V4 Pro 0813 →	1.91 %	$5.58	Cost / Test: $5.58; Input / Output Cost: $1.32 / $3.96; Duration: 2h20m Vals AI · SRE Bench · Accuracy ↗
GPT-5.6 Luna →	1.53 %	$3.44	Cost / Test: $3.44; Input / Output Cost: $0.2 / $1.2; Duration: 1h42m Vals AI · SRE Bench · Accuracy ↗
DeepSeek V4.1 Flash →	0.76 %	$0.55	Cost / Test: $0.55; Input / Output Cost: $0.3 / $1.2; Duration: 1h01m Vals AI · SRE Bench · Accuracy ↗
GLM 5.3 Flash →	0.76 %	$2.29	Cost / Test: $2.29; Input / Output Cost: $0.075 / $0.25; Duration: 3h15m Vals AI · SRE Bench · Accuracy ↗
Grok 4.5 →	0.76 %	$13.42	Cost / Test: $13.42; Input / Output Cost: $2 / $6; Duration: 1h06m Vals AI · SRE Bench · Accuracy ↗
Claude Sonnet 5 →	0.38 %	$18.49	Cost / Test: $18.49; Input / Output Cost: $2 / $10; Duration: 3h53m Vals AI · SRE Bench · Accuracy ↗
Step 5 Preview →	0.38 %	$3.07	Cost / Test: $3.07; Input / Output Cost: $1 / $2.7; Duration: 1h34m Vals AI · SRE Bench · Accuracy ↗
GLM 5.2 →	0 %	$26.58	Cost / Test: $26.58; Input / Output Cost: $1.4 / $4.4; Duration: 2h29m Vals AI · SRE Bench · Accuracy ↗
Muse Spark 1.2 →	0 %	$23.43	Cost / Test: $23.43; Input / Output Cost: $1.25 / $4.25; Duration: 1h40m Vals AI · SRE Bench · Accuracy ↗
Gemini 3.1 Pro Preview (02/26) →	0 %	$11.34	Cost / Test: $11.34; Input / Output Cost: $2 / $12; Duration: 1h03m Vals AI · SRE Bench · Accuracy ↗
Inkling →	0 %	$2.23	Cost / Test: $2.23; Input / Output Cost: $1 / $4.05; Duration: 28m39s Vals AI · SRE Bench · Accuracy ↗

Reported in recent model launches

These model reports include this benchmark. Their harnesses, task subsets, scoring policies, and resource budgets can differ; follow each report’s evaluation notes before joining the numbers.

OpenAI · GPT-6 Astra →

What it measures

In the GPT-6 Astra report, this name refers to software reverse engineering rather than site-reliability incident handling.

How to read the result

Inspect the precise benchmark identity before joining results: an acronym can refer to unrelated evaluation suites.

Limits and comparison traps

Single-attempt and multiple-attempt success are different measurements. Keep the task release and original source attached to every result.

Published evaluation

The linked publication documents the task and scoring procedure. Model results retain the source’s evaluation setting, native score units and available uncertainty.

Where this fits in an evaluation suite

The agent wrapper is part of the system. Pin tools, dependencies, timeouts, tests, and the repository revision. Compare contest coding separately from repository maintenance.

Start with a capability hypothesis, then choose a second test with a different failure mode. Keep the checkpoint, reasoning effort, agent tools, sampling policy, and scoring procedure fixed when measuring a training intervention. Record failures and missing outputs in the denominator.

A reproducible comparison

Pin the benchmark release, task split, model configuration, tools, sampling budget, and evaluation harness described by the primary source.

Publish the exact run configuration, item coverage, score units, and available uncertainty. If a source supplies only an aggregate or a single run, preserve that limit. A new model name or a rounded score alone does not establish a capability improvement.

PRIMARY EVIDENCE & FURTHER READING

SRE Bench — evaluation methodology and results ↗
Vals AI · 2026
GPT-6 Astra: A new generation of intelligence ↗
OpenAI

Reviewed 2026-10-26. Explanation and interpretation by RLTasks. Benchmark ownership remains with its original authors.

Continue exploring

All code & engineering benchmarks →