gpt-4-1106-preview · Aider coding benchmarks · Refactoring · percent correct ↗
OpenAI / MODEL EVIDENCE
GPT-4-1106-preview
7 benchmarks · 24 observations · 6 sources
Scores at a glance
7 benchmarks · large dot: GPT-4-1106-preview; small dots: closest models where reported
HarmBench ↗9.3%
GPT-4-1106-preview · HarmBench · HarmBench harmful compliance · SORRY-Bench paper · cross-benchmark comparison ↗
InFoBench ↗85.5%
gpt-4-1106-preview · InFoBench · Decomposed requirement following rate · GPT-4-1106 evaluator ↗
HumanEval ↗83.7%
gpt-4-1106-preview · HumanEval · Pass@1 · OpenAI Simple Evals · archived July 2025 ↗
gpt-4-1106-preview · DROP (Discrete Reasoning Over Paragraphs) · F1 · three-shot · OpenAI Simple Evals · archived July 2025 ↗
GPT-4-1106-preview · CoCoNot (Contextual Non-Compliance) · No system prompt · Incomplete · compliance rate · reported model comparison ↗
11 more reported results
7.6% · GPT-4-1106-preview · CoCoNot (Contextual Non-Compliance) · No system prompt · Unsupported · compliance rate · reported model comparison · The Art of Saying No: Contextual Noncompliance in Language Models ↗
2.1% · GPT-4-1106-preview · CoCoNot (Contextual Non-Compliance) · No system prompt · Indeterminate · compliance rate · reported model comparison · The Art of Saying No: Contextual Noncompliance in Language Models ↗
2% · GPT-4-1106-preview · CoCoNot (Contextual Non-Compliance) · No system prompt · Safety · compliance rate · reported model comparison · The Art of Saying No: Contextual Noncompliance in Language Models ↗
1.2% · GPT-4-1106-preview · CoCoNot (Contextual Non-Compliance) · No system prompt · Humanizing · compliance rate · reported model comparison · The Art of Saying No: Contextual Noncompliance in Language Models ↗
97.4% · GPT-4-1106-preview · CoCoNot (Contextual Non-Compliance) · No system prompt · Contrast set · compliance rate · reported model comparison · The Art of Saying No: Contextual Noncompliance in Language Models ↗
22.7% · GPT-4-1106-preview · CoCoNot (Contextual Non-Compliance) · With system prompt · Incomplete · compliance rate · reported model comparison · The Art of Saying No: Contextual Noncompliance in Language Models ↗
5.7% · GPT-4-1106-preview · CoCoNot (Contextual Non-Compliance) · With system prompt · Unsupported · compliance rate · reported model comparison · The Art of Saying No: Contextual Noncompliance in Language Models ↗
2.1% · GPT-4-1106-preview · CoCoNot (Contextual Non-Compliance) · With system prompt · Indeterminate · compliance rate · reported model comparison · The Art of Saying No: Contextual Noncompliance in Language Models ↗
1% · GPT-4-1106-preview · CoCoNot (Contextual Non-Compliance) · With system prompt · Safety · compliance rate · reported model comparison · The Art of Saying No: Contextual Noncompliance in Language Models ↗
4.9% · GPT-4-1106-preview · CoCoNot (Contextual Non-Compliance) · With system prompt · Humanizing · compliance rate · reported model comparison · The Art of Saying No: Contextual Noncompliance in Language Models ↗
94.7% · GPT-4-1106-preview · CoCoNot (Contextual Non-Compliance) · With system prompt · Contrast set · compliance rate · reported model comparison · The Art of Saying No: Contextual Noncompliance in Language Models ↗
gpt-4-1106-preview · NYU CTF Bench · pass@1 · 200-challenge full set ↗
6 more reported results
1.9231% · gpt-4-1106-preview · NYU CTF Bench · pass@1 · crypto subset · NYU CTF Bench: A Scalable Open-Source Benchmark Dataset for Evaluating LLMs in Offensive Security — official leaderboard ↗
13.3333% · gpt-4-1106-preview · NYU CTF Bench · pass@1 · forensics subset · NYU CTF Bench: A Scalable Open-Source Benchmark Dataset for Evaluating LLMs in Offensive Security — official leaderboard ↗
16.6667% · gpt-4-1106-preview · NYU CTF Bench · pass@1 · misc subset · NYU CTF Bench: A Scalable Open-Source Benchmark Dataset for Evaluating LLMs in Offensive Security — official leaderboard ↗
5.1282% · gpt-4-1106-preview · NYU CTF Bench · pass@1 · pwn subset · NYU CTF Bench: A Scalable Open-Source Benchmark Dataset for Evaluating LLMs in Offensive Security — official leaderboard ↗
9.8039% · gpt-4-1106-preview · NYU CTF Bench · pass@1 · rev subset · NYU CTF Bench: A Scalable Open-Source Benchmark Dataset for Evaluating LLMs in Offensive Security — official leaderboard ↗
0% · gpt-4-1106-preview · NYU CTF Bench · pass@1 · web subset · NYU CTF Bench: A Scalable Open-Source Benchmark Dataset for Evaluating LLMs in Offensive Security — official leaderboard ↗
Closest models by score
Similar positions on shared benchmark leaderboards. Smaller gaps are closer.
There are not enough shared scored settings to identify close comparisons reliably.
Sources
Every plotted observation links to its evaluation. Reported configurations, benchmark versions and task subsets can differ; similarity describes shared scores, not training lift.
- Aider refactoring benchmark ↗
- InFoBench: Evaluating Instruction Following Ability in Large Language Models — published result table ↗
All 10 authors
Yiwei Qin, Kaiqiang Song, Yebowen Hu, Wenlin Yao, Sangwoo Cho, Xiaoyang Wang, Xuansheng Wu, Fei Liu, Pengfei Liu, Dong Yu
- SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal — published result table ↗
All 16 authors
Tinghao Xie, Xiangyu Qi, Yi Zeng, Yangsibo Huang, Udari Madhushani Sehwag, Kaixuan Huang, Luxi He, Boyi Wei, Dacheng Li, Ying Sheng, Ruoxi Jia, Bo Li, Kai Li, Danqi Chen, Peter Henderson, Prateek Mittal
- The Art of Saying No: Contextual Noncompliance in Language Models — published result table ↗
All 14 authors
Brahman, Faeze, Kumar, Sachin, Balachandran, Vidhisha, Dasigi, Pradeep, Pyatkin, Valentina, Ravichander, Abhilasha, Wiegreffe, Sarah, Dziri, Nouha, Chandu, Khyathi, Hessel, Jack, Tsvetkov, Yulia, Smith, Noah A., Choi, Yejin, Hajishirzi, Hannaneh
- NYU CTF Bench: A Scalable Open-Source Benchmark Dataset for Evaluating LLMs in Offensive Security — official leaderboard ↗
All 13 authors
Minghao Shao, Sofija Jancheska, Meet Udeshi, Brendan Dolan-Gavitt, Haoran Xi, Kimberly Milner, Boyuan Chen, Max Yin, Siddharth Garg, Prashanth Krishnamurthy, Farshad Khorrami, Ramesh Karri, Muhammad Shafique
- Simple Evals — model benchmark results ↗
Names used in source tables
GPT-4-1106-preview · gpt-4-1106-preview