Qwen2-Audio-7B-Inst.

42.6494% · Qwen2-Audio-7B-Inst. · Dynamic-SUPERB · LLM-C classification macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

42.3765% · Qwen2-Audio-7B-Inst. · Dynamic-SUPERB · Environmental audio · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46.2235% · Qwen2-Audio-7B-Inst. · Dynamic-SUPERB · Environmental audio · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

17.1489% · Qwen2-Audio-7B-Inst. · Dynamic-SUPERB · Music & harmony · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

17.4083% · Qwen2-Audio-7B-Inst. · Dynamic-SUPERB · Music & harmony · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

41.4966% · Qwen2-Audio-7B-Inst. · Dynamic-SUPERB · Speech content & structure · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

43.1586% · Qwen2-Audio-7B-Inst. · Dynamic-SUPERB · Speech content & structure · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

60.4305% · Qwen2-Audio-7B-Inst. · Dynamic-SUPERB · Speech recognition & speakers · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

63.656% · Qwen2-Audio-7B-Inst. · Dynamic-SUPERB · Speech recognition & speakers · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

39.3154% · Qwen2-Audio-7B-Inst. · Dynamic-SUPERB · Voice & speech behavior · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

41.7511% · Qwen2-Audio-7B-Inst. · Dynamic-SUPERB · Voice & speech behavior · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

28.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Chord Classification · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

17% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Key Detection (GiantSteps - Key) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

96.76% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Transcription (MAESTRO) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.6875 TER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Transcription (MAESTRO) · TER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

11.8% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Percussion Instruments Tonic Classification (Mridangam Tonic) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

18.11% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Pitch Classification (Nsynth) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

84.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Pitch Extraction By Lyrics (CSD) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

165.96 % TER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Pitch Extraction By Lyrics (CSD) · TER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

8.33% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emotion Classificaiton In Songs (EMOTIFY) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.8% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Emotion Detection (MTG) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

37.69% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Genre Classification (ISMIR04) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.1% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Genre Classification (MTG) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Music Tagging (MagnaTagATune) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.3% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Music Tagging (MTG) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

15.18% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Music Genre Classification (FMA) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

27.12% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Percussion Instruments Classification (Beijing Opera Percussion) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10.8% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Percussion Instruments Stroke Classification (Mridangam Stroke) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

54.41% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Classification (Nsynth) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

24.92% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Combination Recognition (OpenMIC-2018) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

35.22% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Source Classification (Nsynth) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.4% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Instrument Classification (MTG) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.4485 MSE · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · MSE · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.0099 KTAU · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · KTAU · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.0255 LCC · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · LCC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.0124 SRCC · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · SRCC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Beat Tracking (ASAP) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

40.91% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Music Beat Tracking (ASAP) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

76.0904 Miss Time · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Music Beat Tracking (ASAP) · Miss Time · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

68.75% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Editing Identification (People’s Speech) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

154.6202 ACC · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Editing Identification (People’s Speech) · ACC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

54% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Scene Fake Detection (ASPIRE) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

27.9% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Deep Fake Detection (LJSpeech, WaveFake, MUSDB18HQ) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

21.43% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Voice Deepfake Detection (CtrSVDD, ACEKiSing, M4Singer) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Duration Prediction (NTUML2021) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

62.515 MSE · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Duration Prediction (NTUML2021) · MSE · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

89.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Speech Classification (MAESTRO, Librispeech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

19.83% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sound Effect Detection (RemFx) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

56.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Detection (LibriSpeech-TestClean) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

55.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Detection (LibriSpeech-TestOther) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Detection (LJSpeech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

94.93 % WER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Children Song Transcript Verification (CSD) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3.9155 Sacre Bleu · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Lyric Translation (SingSet) · Sacre Bleu · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

115.62 % MER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Song Lyric Recognition (SingSet) · MER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

15.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Vocal Technique Detection (VocalSet) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Segment Retrieval (Clotho) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

7.07 % IoU · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Segment Retrieval (Clotho) · IoU · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

14.29% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Sound Event Detection (DCASE2016 Task2) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

41.55% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-channel Sound Event Understanding (STARSS23) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

34% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Animal Classification (WaveSource-Test) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

79% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Bird Sound Detection (Warblrb10k) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

14% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Cat Emotion Classification (Cat Sound Classification Dataset V2) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Cornell Birdcall Identification · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

36% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Animals) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

42.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Beehive States Classification (Beehive States) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

99.72% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Domestic Environment Sound Event Detection (DESED-PublicEval) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.01 Event-based F1 · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Domestic Environment Sound EventDetection (DESED-PublicEval) · Event-based F1 · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

74.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emergency Traffic Detection (Large-Scale-Audio-dataset) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

25% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Exterior And Urban Noises) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Interior And Domestic Sounds) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Natural Soundscapes And Water Sounds) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4.88% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (UrbanSound8K-Urban Noises) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

63.16% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environment Recognition (ESC50) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

43.9% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Environmental Sound Classification (ESC50) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

11.17% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Vocal Imitation Classification (Vocal Imitations) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

58% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Spatial Distance Prediction (Spatial LibriSpeech) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.6915 MEDAE · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Spatial Distance Prediction (Spatial LibriSpeech) · MEDAE · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

26% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · How Far Are You (3DSpeaker) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

25% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sound Position Prediction · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.1063 Angle Diff · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sound Position Prediction · Angle Diff · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

54% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Emotion Classification (Daily Talk) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.2% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emoji Grounded Speech Emotion Recognition (RAVDESS) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.96% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emotion Change Detection (Ravdess) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

36.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emotion Recognition (MELD) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

61.1% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Emotion Recognition (CREMAD) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

75.83% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Emotion Recognition (RAVDESS) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.72% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Covid19 Cough Audio Classification (CoughVid) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

29% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Human And Non Speech Sounds) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

23.57% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Human Non Speech Sound Recognition (Nonspeech7k, CommonVoice) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

87.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Human Screaming Detection (Environmentdb) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

30.97% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Vocal Sound Recognition (VocalSound) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.73% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Segment Counting (Librispeech-words) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13.46% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Segment Counting (Librispeech-words) · ACC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3.0622 Abs Diff · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Segment Counting (Librispeech-words) · Abs Diff · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

16.11% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phone Segment Counting (VoxAngeles) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

6.6% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phone Segment Counting (VoxAngeles) · ACC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.8706 Abs Diff · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phone Segment Counting (VoxAngeles) · Abs Diff · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100.96 % PER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Phoneme Recognition (LibriSpeech-TestClean) · PER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100.88 % PER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Recognition (LibriSpeech-TestOther) · PER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3.24% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Consonant Place Of Articulation) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

6.74% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Manner Of Articulation) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3.97% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Phone) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

57.23% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Vowel Frontness) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

36.86% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Vowel Height) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

69.65% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Vowel Roundedness) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

44% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Heteronym Differentiation (HeteronymEn) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Accuracy (speechocean762-Ranking) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.94% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Accuracy (speechocean762-scoring) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

-0.0159 PCC · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Accuracy (speechocean762-scoring) · PCC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

44.7% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-lingual Pronunciation Similarity (VoxAngeles) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

14% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Accent Classification (Accentdb Extended) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

23.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Stress Detection (MIRSD) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

84.38% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Third Tone Sandhi Recognition (NCCU Corpus of Spoken Taiwan Mandarin) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0 IoU · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Third Tone Sandhi Recognition (NCCU Corpus of Spoken Taiwan Mandarin) · IoU · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50.56% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Fluency (speechocean762-Ranking) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.81% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Fluency (speechocean762-Scoring) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

-0.0858 PCC · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Fluency (speechocean762-Scoring) · PCC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

51.11% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Prosodic (speechocean762-Ranking) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3.1% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Prosodic (speechocean762-Scoring) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.0201 PCC · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Prosodic (speechocean762-Scoring) · PCC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

51.74% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Prosody Naturalness (ProsAudit-Lexical) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50.76% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Prosody Naturalness (ProsAudit-Protosyntax) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

23% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoof Detection (ASVspoof2015) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

32.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoof Detection (ASVspoof2017) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Deep Fake Voice Recognition (DEEP-VOICE) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

56.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Enhancement Detection (LibriTTS-TestClean, WHAM) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Fraud Robocall Recognition (CallHome) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

63.16% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Fraud Robocall Recognition (Promo) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

61.54% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Fraud Robocall Recognition (Robocall) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

36.11% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Language Identification (VoxLingua107) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

93% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Language Identification (VoxForge) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

35.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Age Classification (Common Voice) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

97.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Gender Recognition by Voice (Common Voice) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.4% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Speaker Count Identification (LibriCount) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13.76% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Speaker Count Identification (LibriCount) · ACC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

51% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Speaker Detection (VCTK) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

26.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Counting (LibriTTS-TestClean) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

52% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Verification (LibriSpeech-TestClean) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

54% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Verification (LibriSpeech-TestOther) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

51% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Verification (VCTK) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

98% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Speaker Diarization (Libri2Mix-Test) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

93.29 % DER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Speaker Diarization (Libri2Mix-Test) · DER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

51% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Speaker Verification (SUPERB Hidden Set) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Gaussian) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

47% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Music) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Noise) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

47% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Speech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

54.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Music) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

61% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Noise) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

57% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Speech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

54.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Gaussian) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

17.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Gaussian) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

14% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Music) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

19% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Noise) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

20.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Speech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

48% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (LJSpeech, RirsNoises-LargeRoom) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

48% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (LJSpeech, RirsNoises-MediumRoom) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

48% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (LJSpeech, RirsNoises-SmallRoom) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (VCTK, RirsNoises-LargeRoom) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (VCTK, RirsNoises-MediumRoom) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (VCTK, RirsNoises-SmallRoom) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

29% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · N-Best Correction (Librispeech-TestOther) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

38.55 % WER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · AAVE Speech Recognition (CORAAL) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

116.88 % MER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-switch Speech Recognition (NTUML2021) · MER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-Switching Speech Recognition (ASCEND) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

60.8% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-Switching Speech Recognition (ASCEND) · ACC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

24.96 % WER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-de) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

17.47 % WER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-en) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

18.19 % WER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-es) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

19.01 % WER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-fr) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

33.16 % WER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-it) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

42.94 % WER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-nl) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

101.41 % WER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-pl) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

24.72 % WER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-pt) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

38.57 % WER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · PTBR Speech Recognition (Common Voice) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

36.7 % WER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB ASR (LibriSpeech-TestClean) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

40.28 % WER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · ASR (LibriSpeech-TestOther) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

178.21 % WER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Ar) (Common Voice) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

75.08 % WER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Es) (Common Voice) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

80.21 % WER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Spontaneous) (CHIME6-Test) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

270.63 % CER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Zh) (Common Voice) · CER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

132.13 % WER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Target Speaker ASR (AMI-test) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

52% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Speaker Detection (LibriSpeech-TestClean) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Command Recognition (AudioMNIST) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

77.07% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Command Recognition (AudioMNIST) · ACC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

92% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Text Matching (LibriSpeech-TestClean) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

92.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Text Matching (LibriSpeech-TestOther) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

90% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Text Matching (LJSpeech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

61.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Term Detection (LibriSpeech-TestClean) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

64% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Term Detection (LibriSpeech-TestOther) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

79.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Term Detection (LJSpeech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

47% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Keyword Spotting (Speech Commands V1-Test) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

53.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Query by Example (Quesst14-Eval) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

52.4% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Stuttering Detection (SEP28k) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

16.35% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Voice Disorder Classification (VOICED) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

66.67% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Conversation Matching · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

15.83% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Act Classification (SLUE-HVB) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

30% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Act Classification (Daily Talk) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Act Pairing (Daily Talk) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

26% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Intent Classification (SLURP) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

68.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Intent Classification (SLURP-Action) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

56% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Intent Classification (SLURP-Intent) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

25% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Named Entity Localization (SLUE-VoxPopuli) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.1063 F1 · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Named Entity Localization (SLUE-VoxPopuli) · F1 · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

91.3% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Named Entity Recognition (SLUE-VoxPopuli) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.0625 IoU · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Named Entity Recognition (SLUE-VoxPopuli) · IoU · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

48% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sarcasm Detection (Mustard) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46.8% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Semantic Textual Similarity (SpokenSTS) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Sentiment Analysis (MELD) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

43.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Digit Arithmetic (AudioMNIST) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Slot Filling (AudioSnips-Test) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.7704 Slot Type F1 · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Slot Filling (AudioSnips-Test) · Slot Type F1 · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.3065 Slot Value CER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Slot Filling (AudioSnips-Test) · Slot Value CER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-Switching Semantic Grammar Acceptability Comparison (CSZS-zh-en) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50.14% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Nonce Word Detection (sWUGGY) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.3091 POS · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · PoS Estimation (LibriTTS) · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.7514 POS · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · PoS Estimation with transcription (LibriTTS) · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49.4% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sentence Grammar Acceptability (sBLIMP) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

23.3458 Sacre Bleu · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Speech Translation (CoVoST2-Test) · Sacre Bleu · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

16% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Chord Classification · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Key Detection (GiantSteps - Key) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

96.22% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Transcription (MAESTRO) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100 % TER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Transcription (MAESTRO) · TER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13.2% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Percussion Instruments Tonic Classification (Mridangam Tonic) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

18.9% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Pitch Classification (Nsynth) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

73% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Pitch Extraction By Lyrics (CSD) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.42 TER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Pitch Extraction By Lyrics (CSD) · TER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

8.3% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emotion Classificaiton In Songs (EMOTIFY) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3.3% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Emotion Detection (MTG) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

37.19% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Genre Classification (ISMIR04) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Genre Classification (MTG) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Music Tagging (MagnaTagATune) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4.8% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Music Tagging (MTG) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

17% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Music Genre Classification (FMA) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

29.26% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Percussion Instruments Classification (Beijing Opera Percussion) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

11.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Percussion Instruments Stroke Classification (Mridangam Stroke) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

55.4% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Classification (Nsynth) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

25.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Combination Recognition (OpenMIC-2018) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

37.9% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Source Classification (Nsynth) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.6% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Instrument Classification (MTG) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.83% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.381 MSE · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · MSE · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.018 KTAU · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · KTAU · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.04 LCC · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · LCC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.023 SRCC · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · SRCC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Beat Tracking (ASAP) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

40.91% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Music Beat Tracking (ASAP) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

31.399 Miss Time · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Music Beat Tracking (ASAP) · Miss Time · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

31.25% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Editing Identification (People’s Speech) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

165.068 ACC · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Editing Identification (People’s Speech) · ACC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

54% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Scene Fake Detection (ASPIRE) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

39.7% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Deep Fake Detection (LJSpeech, WaveFake, MUSDB18HQ) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

31.9% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Voice Deepfake Detection (CtrSVDD, ACEKiSing, M4Singer) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Duration Prediction (NTUML2021) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

62.155 MSE · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Duration Prediction (NTUML2021) · MSE · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

89.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Speech Classification (MAESTRO, Librispeech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

19.67% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sound Effect Detection (RemFx) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

56.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Detection (LibriSpeech-TestClean) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

55.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Detection (LibriSpeech-TestOther) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

47.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Detection (LJSpeech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

94.9 % WER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Children Song Transcript Verification (CSD) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3.916 Sacre Bleu · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Lyric Translation (SingSet) · Sacre Bleu · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.1562 MER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Song Lyric Recognition (SingSet) · MER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

15.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Vocal Technique Detection (VocalSet) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

28% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Segment Retrieval (Clotho) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.193 IoU · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Segment Retrieval (Clotho) · IoU · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

7.1% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Sound Event Detection (DCASE2016 Task2) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

43% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-channel Sound Event Understanding (STARSS23) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

34.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Animal Classification (WaveSource-Test) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

82.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Bird Sound Detection (Warblrb10k) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

16% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Cat Emotion Classification (Cat Sound Classification Dataset V2) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Cornell Birdcall Identification · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

34% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Animals) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

92% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Beehive States Classification (Beehive States) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

99.72% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Domestic Environment Sound Event Detection (DESED-PublicEval) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1 Event-based F1 · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Domestic Environment Sound EventDetection (DESED-PublicEval) · Event-based F1 · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

75.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emergency Traffic Detection (Large-Scale-Audio-dataset) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

25% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Exterior And Urban Noises) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

15% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Interior And Domestic Sounds) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

8% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Natural Soundscapes And Water Sounds) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

7.9% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (UrbanSound8K-Urban Noises) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

66.7% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environment Recognition (ESC50) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

52% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Environmental Sound Classification (ESC50) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

11.33% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Vocal Imitation Classification (Vocal Imitations) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

41.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Spatial Distance Prediction (Spatial LibriSpeech) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.784 MEDAE · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Spatial Distance Prediction (Spatial LibriSpeech) · MEDAE · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

26.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · How Far Are You (3DSpeaker) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sound Position Prediction · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

54% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Emotion Classification (Daily Talk) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.8% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emoji Grounded Speech Emotion Recognition (RAVDESS) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10.3% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emotion Change Detection (Ravdess) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

36.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emotion Recognition (MELD) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

62.4% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Emotion Recognition (CREMAD) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

76.25% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Emotion Recognition (RAVDESS) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

8.3% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Covid19 Cough Audio Classification (CoughVid) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

29% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Human And Non Speech Sounds) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

23.6% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Human Non Speech Sound Recognition (Nonspeech7k, CommonVoice) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

87.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Human Screaming Detection (Environmentdb) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

38.3% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Vocal Sound Recognition (VocalSound) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13.44% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Segment Counting (Librispeech-words) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.153 ACC · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Segment Counting (Librispeech-words) · ACC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.99 Abs Diff · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Segment Counting (Librispeech-words) · Abs Diff · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

28.23% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phone Segment Counting (VoxAngeles) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.095 ACC · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phone Segment Counting (VoxAngeles) · ACC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.722 Abs Diff · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phone Segment Counting (VoxAngeles) · Abs Diff · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100.96 % PER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Phoneme Recognition (LibriSpeech-TestClean) · PER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100.88 % PER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Recognition (LibriSpeech-TestOther) · PER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

9.9% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Consonant Place Of Articulation) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

6.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Manner Of Articulation) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

5.2% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Phone) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

58.2% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Vowel Frontness) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

38.7% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Vowel Height) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

70.1% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Vowel Roundedness) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

44% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Heteronym Differentiation (HeteronymEn) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Accuracy (speechocean762-Ranking) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.94% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Accuracy (speechocean762-scoring) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

-0.011 PCC · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Accuracy (speechocean762-scoring) · PCC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

43.2% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-lingual Pronunciation Similarity (VoxAngeles) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

14% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Accent Classification (Accentdb Extended) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

20% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Stress Detection (MIRSD) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3.12% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Third Tone Sandhi Recognition (NCCU Corpus of Spoken Taiwan Mandarin) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.065 IoU · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Third Tone Sandhi Recognition (NCCU Corpus of Spoken Taiwan Mandarin) · IoU · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

54.2% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Fluency (speechocean762-Ranking) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.4% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Fluency (speechocean762-Scoring) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

-0.086 PCC · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Fluency (speechocean762-Scoring) · PCC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

47.2% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Prosodic (speechocean762-Ranking) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.69% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Prosodic (speechocean762-Scoring) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

-0.017 PCC · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Prosodic (speechocean762-Scoring) · PCC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

58.7% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Prosody Naturalness (ProsAudit-Lexical) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

59.2% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Prosody Naturalness (ProsAudit-Protosyntax) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

24% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoof Detection (ASVspoof2015) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

35.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoof Detection (ASVspoof2017) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

47% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Deep Fake Voice Recognition (DEEP-VOICE) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

58% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Enhancement Detection (LibriTTS-TestClean, WHAM) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Fraud Robocall Recognition (CallHome) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

63.2% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Fraud Robocall Recognition (Promo) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

61.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Fraud Robocall Recognition (Robocall) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

54.32% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Language Identification (VoxLingua107) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

93% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Language Identification (VoxForge) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

36.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Age Classification (Common Voice) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

97.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Gender Recognition by Voice (Common Voice) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

9.6% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Speaker Count Identification (LibriCount) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

15.6% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Speaker Count Identification (LibriCount) · ACC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

51% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Speaker Detection (VCTK) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

26.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Counting (LibriTTS-TestClean) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

63% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Verification (LibriSpeech-TestClean) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

70% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Verification (LibriSpeech-TestOther) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

60.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Verification (VCTK) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Speaker Diarization (Libri2Mix-Test) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

59% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Speaker Verification (SUPERB Hidden Set) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Gaussian) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

47.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Music) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

59.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Noise) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

47% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Speech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

58.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Music) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

67% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Noise) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

57% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Speech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

54.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Gaussian) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

22% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Gaussian) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

15% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Music) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

21% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Noise) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

21.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Speech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

48% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (LJSpeech, RirsNoises-LargeRoom) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (LJSpeech, RirsNoises-MediumRoom) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

51% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (LJSpeech, RirsNoises-SmallRoom) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (VCTK, RirsNoises-LargeRoom) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

51.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (VCTK, RirsNoises-MediumRoom) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

51.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (VCTK, RirsNoises-SmallRoom) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

41.8% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · N-Best Correction (Librispeech-TestOther) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

38.6 % WER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · AAVE Speech Recognition (CORAAL) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

116.88 % MER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-switch Speech Recognition (NTUML2021) · MER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-Switching Speech Recognition (ASCEND) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

60.8% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-Switching Speech Recognition (ASCEND) · ACC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

24.96 % WER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-de) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

17.47 % WER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-en) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

18.19 % WER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-es) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

19.01 % WER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-fr) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

33.16 % WER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-it) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

42.94 % WER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-nl) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

101.41 % WER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-pl) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

24.72 % WER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-pt) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

38.57 % WER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · PTBR Speech Recognition (Common Voice) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

36.7 % WER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB ASR (LibriSpeech-TestClean) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

40.28 % WER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · ASR (LibriSpeech-TestOther) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

178.21 % WER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Ar) (Common Voice) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

75.08 % WER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Es) (Common Voice) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

80.21 % WER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Spontaneous) (CHIME6-Test) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

270.63 % CER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Zh) (Common Voice) · CER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

132.13 % WER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Target Speaker ASR (AMI-test) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

52% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Speaker Detection (LibriSpeech-TestClean) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Command Recognition (AudioMNIST) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.771 ACC · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Command Recognition (AudioMNIST) · ACC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

84.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Text Matching (LibriSpeech-TestClean) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

87.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Text Matching (LibriSpeech-TestOther) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

76% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Text Matching (LJSpeech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

62% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Term Detection (LibriSpeech-TestClean) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

66% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Term Detection (LibriSpeech-TestOther) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

79.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Term Detection (LJSpeech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

59% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Keyword Spotting (Speech Commands V1-Test) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

53.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Query by Example (Quesst14-Eval) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

52.6% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Stuttering Detection (SEP28k) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

17.3% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Voice Disorder Classification (VOICED) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

61.1% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Conversation Matching · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

20.6% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Act Classification (SLUE-HVB) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

30.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Act Classification (Daily Talk) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

48.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Act Pairing (Daily Talk) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

26.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Intent Classification (SLURP) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

67.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Intent Classification (SLURP-Action) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

43% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Intent Classification (SLURP-Intent) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Named Entity Localization (SLUE-VoxPopuli) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

64.13% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Named Entity Recognition (SLUE-VoxPopuli) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.119 IoU · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Named Entity Recognition (SLUE-VoxPopuli) · IoU · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

48% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sarcasm Detection (Mustard) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49.2% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Semantic Textual Similarity (SpokenSTS) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Sentiment Analysis (MELD) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

43.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Digit Arithmetic (AudioMNIST) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

58% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Slot Filling (AudioSnips-Test) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.927 Slot Type F1 · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Slot Filling (AudioSnips-Test) · Slot Type F1 · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.239 Slot Value CER · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Slot Filling (AudioSnips-Test) · Slot Value CER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49.5% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-Switching Semantic Grammar Acceptability Comparison (CSZS-zh-en) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

61.8% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Nonce Word Detection (sWUGGY) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.3091 POS · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · PoS Estimation (LibriTTS) · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.7514 POS · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · PoS Estimation with transcription (LibriTTS) · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

73.8% · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sentence Grammar Acceptability (sBLIMP) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

23.3458 Sacre Bleu · Qwen2-Audio-7B-Inst. · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Speech Translation (CoVoST2-Test) · Sacre Bleu · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗