vicuna · Do-Not-Answer & BeaverTails · human-rated safety · Original safeguard evaluation ↗
Reported model / MODEL EVIDENCE
vicuna
3 benchmarks · 17 observations · 3 sources
Scores at a glance
3 benchmarks · large dot: vicuna; small dots: closest models where reported
Vicuna · JailbreakBench (JBB-Behaviors) · JBB original-source attack success · source-subset comparison ↗
3 more reported results
80% · Vicuna · JailbreakBench (JBB-Behaviors) · JBB original-source attack success · source-subset comparison · JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models ↗
84% · Vicuna · JailbreakBench (JBB-Behaviors) · JBB original-source attack success · source-subset comparison · JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models ↗
82% · Vicuna · JailbreakBench (JBB-Behaviors) · JBB original-source attack success · source-subset comparison · JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models ↗
Vicuna · CoCoNot (Contextual Non-Compliance) · No system prompt · Incomplete · compliance rate · reported model comparison ↗
11 more reported results
22.9% · Vicuna · CoCoNot (Contextual Non-Compliance) · No system prompt · Unsupported · compliance rate · reported model comparison · The Art of Saying No: Contextual Noncompliance in Language Models ↗
4.9% · Vicuna · CoCoNot (Contextual Non-Compliance) · No system prompt · Indeterminate · compliance rate · reported model comparison · The Art of Saying No: Contextual Noncompliance in Language Models ↗
14.7% · Vicuna · CoCoNot (Contextual Non-Compliance) · No system prompt · Safety · compliance rate · reported model comparison · The Art of Saying No: Contextual Noncompliance in Language Models ↗
20.7% · Vicuna · CoCoNot (Contextual Non-Compliance) · No system prompt · Humanizing · compliance rate · reported model comparison · The Art of Saying No: Contextual Noncompliance in Language Models ↗
91.8% · Vicuna · CoCoNot (Contextual Non-Compliance) · No system prompt · Contrast set · compliance rate · reported model comparison · The Art of Saying No: Contextual Noncompliance in Language Models ↗
24.4% · Vicuna · CoCoNot (Contextual Non-Compliance) · With system prompt · Incomplete · compliance rate · reported model comparison · The Art of Saying No: Contextual Noncompliance in Language Models ↗
13.4% · Vicuna · CoCoNot (Contextual Non-Compliance) · With system prompt · Unsupported · compliance rate · reported model comparison · The Art of Saying No: Contextual Noncompliance in Language Models ↗
2.1% · Vicuna · CoCoNot (Contextual Non-Compliance) · With system prompt · Indeterminate · compliance rate · reported model comparison · The Art of Saying No: Contextual Noncompliance in Language Models ↗
8.9% · Vicuna · CoCoNot (Contextual Non-Compliance) · With system prompt · Safety · compliance rate · reported model comparison · The Art of Saying No: Contextual Noncompliance in Language Models ↗
8.5% · Vicuna · CoCoNot (Contextual Non-Compliance) · With system prompt · Humanizing · compliance rate · reported model comparison · The Art of Saying No: Contextual Noncompliance in Language Models ↗
88.7% · Vicuna · CoCoNot (Contextual Non-Compliance) · With system prompt · Contrast set · compliance rate · reported model comparison · The Art of Saying No: Contextual Noncompliance in Language Models ↗
Closest models by score
Similar positions on shared benchmark leaderboards. Smaller gaps are closer.
GPT-4 →OpenAI
3 of 3 shared benchmark comparisons
How similarity is measured
Mean absolute gap in percentile positions on shared benchmark settings; lower is closer. At least 60% overlap is required, with a minimum of three when available. Independent evaluations are used when there are at least three; otherwise published report comparisons are used. Repeated model observations within one setting are averaged before ranking. Writing-style traits and composite indices are excluded. This describes similarity, not an overall quality ranking.
Sources
Every plotted observation links to its evaluation. Reported configurations, benchmark versions and task subsets can differ; similarity describes shared scores, not training lift.
- JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models — published result table ↗
All 12 authors
Patrick Chao, Edoardo Debenedetti, Alexander Robey, Maksym Andriushchenko, Francesco Croce, Vikash Sehwag, Edgar Dobriban, Nicolas Flammarion, George J. Pappas, Florian Tramer, Hamed Hassani, Eric Wong
- The Art of Saying No: Contextual Noncompliance in Language Models — published result table ↗
All 14 authors
Brahman, Faeze, Kumar, Sachin, Balachandran, Vidhisha, Dasigi, Pradeep, Pyatkin, Valentina, Ravichander, Abhilasha, Wiegreffe, Sarah, Dziri, Nouha, Chandu, Khyathi, Hessel, Jack, Tsvetkov, Yulia, Smith, Noah A., Choi, Yejin, Hajishirzi, Hannaneh
- Do-Not-Answer: Evaluating Safeguards in LLMs ↗
Names used in source tables
Vicuna · vicuna