MU-LLaMA

28.6393% · MU-LLaMA · Dynamic-SUPERB · LLM-C classification macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

23.5118% · MU-LLaMA · Dynamic-SUPERB · Environmental audio · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

25.0982% · MU-LLaMA · Dynamic-SUPERB · Environmental audio · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

11.9111% · MU-LLaMA · Dynamic-SUPERB · Music & harmony · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13.575% · MU-LLaMA · Dynamic-SUPERB · Music & harmony · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

26.1062% · MU-LLaMA · Dynamic-SUPERB · Speech content & structure · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

31.7483% · MU-LLaMA · Dynamic-SUPERB · Speech content & structure · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

30.99% · MU-LLaMA · Dynamic-SUPERB · Speech recognition & speakers · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

35.297% · MU-LLaMA · Dynamic-SUPERB · Speech recognition & speakers · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

22.7963% · MU-LLaMA · Dynamic-SUPERB · Voice & speech behavior · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

30.988% · MU-LLaMA · Dynamic-SUPERB · Voice & speech behavior · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

51% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Chord Classification · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Key Detection (GiantSteps - Key) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Transcription (MAESTRO) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

12.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Percussion Instruments Tonic Classification (Mridangam Tonic) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.33% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Pitch Classification (Nsynth) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

98% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Pitch Extraction By Lyrics (CSD) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100 % TER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Pitch Extraction By Lyrics (CSD) · TER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.67% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emotion Classificaiton In Songs (EMOTIFY) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.8% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Emotion Detection (MTG) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

27.64% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Genre Classification (ISMIR04) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.2% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Genre Classification (MTG) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Music Tagging (MagnaTagATune) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Music Tagging (MTG) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

9.82% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Music Genre Classification (FMA) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

22.03% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Percussion Instruments Classification (Beijing Opera Percussion) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10.3% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Percussion Instruments Stroke Classification (Mridangam Stroke) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

6.99% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Classification (Nsynth) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

19.52% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Combination Recognition (OpenMIC-2018) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

38% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Source Classification (Nsynth) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.6% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Instrument Classification (MTG) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

97.34% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4.53 MSE · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · MSE · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.1705 KTAU · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · KTAU · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.1866 LCC · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · LCC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.2281 SRCC · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · SRCC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Beat Tracking (ASAP) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

80% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Music Beat Tracking (ASAP) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

240 Miss Time · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Music Beat Tracking (ASAP) · Miss Time · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

65.62% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Editing Identification (People’s Speech) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

164.0558 ACC · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Editing Identification (People’s Speech) · ACC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

20% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Scene Fake Detection (ASPIRE) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

39.1% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Deep Fake Detection (LJSpeech, WaveFake, MUSDB18HQ) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

20.95% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Voice Deepfake Detection (CtrSVDD, ACEKiSing, M4Singer) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Duration Prediction (NTUML2021) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1527.785 MSE · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Duration Prediction (NTUML2021) · MSE · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

51.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Speech Classification (MAESTRO, Librispeech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

17% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sound Effect Detection (RemFx) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Detection (LibriSpeech-TestClean) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Detection (LibriSpeech-TestOther) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Detection (LJSpeech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100 % WER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Children Song Transcript Verification (CSD) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.1593 Sacre Bleu · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Lyric Translation (SingSet) · Sacre Bleu · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

117.53 % MER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Song Lyric Recognition (SingSet) · MER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

8.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Vocal Technique Detection (VocalSet) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

82% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Segment Retrieval (Clotho) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0 % IoU · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Segment Retrieval (Clotho) · IoU · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

21.43% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Sound Event Detection (DCASE2016 Task2) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4.93% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-channel Sound Event Understanding (STARSS23) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4.25% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Animal Classification (WaveSource-Test) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

40.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Bird Sound Detection (Warblrb10k) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Cat Emotion Classification (Cat Sound Classification Dataset V2) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3.33% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Cornell Birdcall Identification · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Animals) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

37.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Beehive States Classification (Beehive States) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Domestic Environment Sound Event Detection (DESED-PublicEval) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0 Event-based F1 · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Domestic Environment Sound EventDetection (DESED-PublicEval) · Event-based F1 · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

43.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emergency Traffic Detection (Large-Scale-Audio-dataset) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Exterior And Urban Noises) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Interior And Domestic Sounds) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Natural Soundscapes And Water Sounds) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (UrbanSound8K-Urban Noises) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10.53% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environment Recognition (ESC50) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.4% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Environmental Sound Classification (ESC50) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3.83% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Vocal Imitation Classification (Vocal Imitations) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

97% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Spatial Distance Prediction (Spatial LibriSpeech) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.9212 MEDAE · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Spatial Distance Prediction (Spatial LibriSpeech) · MEDAE · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

18% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · How Far Are You (3DSpeaker) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sound Position Prediction · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

19% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Emotion Classification (Daily Talk) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.9% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emoji Grounded Speech Emotion Recognition (RAVDESS) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.58% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emotion Change Detection (Ravdess) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emotion Recognition (MELD) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

12.4% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Emotion Recognition (CREMAD) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

12.92% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Emotion Recognition (RAVDESS) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

21.91% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Covid19 Cough Audio Classification (CoughVid) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Human And Non Speech Sounds) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

17.14% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Human Non Speech Sound Recognition (Nonspeech7k, CommonVoice) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

42.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Human Screaming Detection (Environmentdb) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.28% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Vocal Sound Recognition (VocalSound) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.67% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Segment Counting (Librispeech-words) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.47% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Segment Counting (Librispeech-words) · ACC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

15.6777 Abs Diff · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Segment Counting (Librispeech-words) · Abs Diff · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

41.47% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phone Segment Counting (VoxAngeles) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

9.14% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phone Segment Counting (VoxAngeles) · ACC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

5.4242 Abs Diff · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phone Segment Counting (VoxAngeles) · Abs Diff · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

110.27 % PER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Phoneme Recognition (LibriSpeech-TestClean) · PER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

111.18 % PER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Recognition (LibriSpeech-TestOther) · PER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.54% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Consonant Place Of Articulation) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

7.96% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Manner Of Articulation) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Phone) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

42.97% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Vowel Frontness) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

39.1% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Vowel Height) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

21.79% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Vowel Roundedness) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

30% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Heteronym Differentiation (HeteronymEn) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

48.06% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Accuracy (speechocean762-Ranking) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

90.47% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Accuracy (speechocean762-scoring) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.0727 PCC · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Accuracy (speechocean762-scoring) · PCC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

25.1% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-lingual Pronunciation Similarity (VoxAngeles) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

27% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Accent Classification (Accentdb Extended) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Stress Detection (MIRSD) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

96.88% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Third Tone Sandhi Recognition (NCCU Corpus of Spoken Taiwan Mandarin) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0 IoU · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Third Tone Sandhi Recognition (NCCU Corpus of Spoken Taiwan Mandarin) · IoU · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49.44% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Fluency (speechocean762-Ranking) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

63.53% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Fluency (speechocean762-Scoring) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.0505 PCC · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Fluency (speechocean762-Scoring) · PCC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49.72% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Prosodic (speechocean762-Ranking) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

94.62% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Prosodic (speechocean762-Scoring) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

-0.1535 PCC · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Prosodic (speechocean762-Scoring) · PCC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

47.1% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Prosody Naturalness (ProsAudit-Lexical) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49.24% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Prosody Naturalness (ProsAudit-Protosyntax) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

69.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoof Detection (ASVspoof2015) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

64% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoof Detection (ASVspoof2017) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Deep Fake Voice Recognition (DEEP-VOICE) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Enhancement Detection (LibriTTS-TestClean, WHAM) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

26.67% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Fraud Robocall Recognition (CallHome) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

5.26% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Fraud Robocall Recognition (Promo) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

30.77% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Fraud Robocall Recognition (Robocall) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Language Identification (VoxLingua107) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

6% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Language Identification (VoxForge) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

23.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Age Classification (Common Voice) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

51% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Gender Recognition by Voice (Common Voice) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

23.2% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Speaker Count Identification (LibriCount) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13.8% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Speaker Count Identification (LibriCount) · ACC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

39% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Speaker Detection (VCTK) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Counting (LibriTTS-TestClean) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

24% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Verification (LibriSpeech-TestClean) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

33% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Verification (LibriSpeech-TestOther) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

19.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Verification (VCTK) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Speaker Diarization (Libri2Mix-Test) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

44% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Speaker Verification (SUPERB Hidden Set) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

41% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Gaussian) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

33% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Music) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

48% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Noise) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Speech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

37.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Music) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

54% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Noise) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

48.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Speech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

38.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Gaussian) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

15% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Gaussian) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

9% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Music) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

11% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Noise) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Speech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

17.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (LJSpeech, RirsNoises-LargeRoom) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (LJSpeech, RirsNoises-MediumRoom) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

14% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (LJSpeech, RirsNoises-SmallRoom) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

14.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (VCTK, RirsNoises-LargeRoom) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

14% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (VCTK, RirsNoises-MediumRoom) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (VCTK, RirsNoises-SmallRoom) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

20.8% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · N-Best Correction (Librispeech-TestOther) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

136.5 % WER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · AAVE Speech Recognition (CORAAL) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

130.18 % MER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-switch Speech Recognition (NTUML2021) · MER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

69.4% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-Switching Speech Recognition (ASCEND) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13.73% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-Switching Speech Recognition (ASCEND) · ACC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

99.37 % WER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-de) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

96.03 % WER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-en) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

103.19 % WER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-es) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

102.25 % WER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-fr) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

102.72 % WER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-it) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

102.05 % WER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-nl) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100.23 % WER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-pl) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

99.71 % WER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-pt) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

639.12 % WER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · PTBR Speech Recognition (Common Voice) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

103.67 % WER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB ASR (LibriSpeech-TestClean) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

111.03 % WER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · ASR (LibriSpeech-TestOther) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

225.28 % WER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Ar) (Common Voice) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

141.68 % WER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Es) (Common Voice) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

137.25 % WER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Spontaneous) (CHIME6-Test) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

435.33 % CER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Zh) (Common Voice) · CER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

108.32 % WER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Target Speaker ASR (AMI-test) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

42.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Speaker Detection (LibriSpeech-TestClean) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.6% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Command Recognition (AudioMNIST) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

9.89% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Command Recognition (AudioMNIST) · ACC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Text Matching (LibriSpeech-TestClean) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

47.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Text Matching (LibriSpeech-TestOther) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Text Matching (LJSpeech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

30.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Term Detection (LibriSpeech-TestClean) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

41% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Term Detection (LibriSpeech-TestOther) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

38.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Term Detection (LJSpeech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Keyword Spotting (Speech Commands V1-Test) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

51% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Query by Example (Quesst14-Eval) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49.6% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Stuttering Detection (SEP28k) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

21.15% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Voice Disorder Classification (VOICED) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3.7% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Conversation Matching · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.83% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Act Classification (SLUE-HVB) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

18.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Act Classification (Daily Talk) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

40% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Act Pairing (Daily Talk) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Intent Classification (SLURP) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

6.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Intent Classification (SLURP-Action) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Intent Classification (SLURP-Intent) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Named Entity Localization (SLUE-VoxPopuli) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

97.83% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Named Entity Recognition (SLUE-VoxPopuli) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0 IoU · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Named Entity Recognition (SLUE-VoxPopuli) · IoU · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

43% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sarcasm Detection (Mustard) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

51.2% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Semantic Textual Similarity (SpokenSTS) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

27.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Sentiment Analysis (MELD) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Digit Arithmetic (AudioMNIST) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

99% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Slot Filling (AudioSnips-Test) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1 Slot Type F1 · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Slot Filling (AudioSnips-Test) · Slot Type F1 · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.587 Slot Value CER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Slot Filling (AudioSnips-Test) · Slot Value CER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

28% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-Switching Semantic Grammar Acceptability Comparison (CSZS-zh-en) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

35.04% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Nonce Word Detection (sWUGGY) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.5327 POS · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · PoS Estimation (LibriTTS) · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.2394 POS · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · PoS Estimation with transcription (LibriTTS) · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50.79% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sentence Grammar Acceptability (sBLIMP) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.0566 Sacre Bleu · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Speech Translation (CoVoST2-Test) · Sacre Bleu · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Chord Classification · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Key Detection (GiantSteps - Key) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Transcription (MAESTRO) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

14.3% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Percussion Instruments Tonic Classification (Mridangam Tonic) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.4% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Pitch Classification (Nsynth) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

97.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Pitch Extraction By Lyrics (CSD) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1 TER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Pitch Extraction By Lyrics (CSD) · TER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.7% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emotion Classificaiton In Songs (EMOTIFY) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

7.2% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Emotion Detection (MTG) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

26.13% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Genre Classification (ISMIR04) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.8% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Genre Classification (MTG) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

9% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Music Tagging (MagnaTagATune) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3.8% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Music Tagging (MTG) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

9.8% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Music Genre Classification (FMA) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

22.02% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Percussion Instruments Classification (Beijing Opera Percussion) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10.6% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Percussion Instruments Stroke Classification (Mridangam Stroke) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10.9% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Classification (Nsynth) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

22.8% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Combination Recognition (OpenMIC-2018) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

41.4% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Source Classification (Nsynth) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Instrument Classification (MTG) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

97.84% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4.372 MSE · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · MSE · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.164 KTAU · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · KTAU · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.098 LCC · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · LCC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.187 SRCC · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · SRCC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Beat Tracking (ASAP) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

78.18% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Music Beat Tracking (ASAP) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

199.877 Miss Time · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Music Beat Tracking (ASAP) · Miss Time · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

87.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Editing Identification (People’s Speech) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

154.109 ACC · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Editing Identification (People’s Speech) · ACC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

19% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Scene Fake Detection (ASPIRE) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

40.3% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Deep Fake Detection (LJSpeech, WaveFake, MUSDB18HQ) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

40.8% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Voice Deepfake Detection (CtrSVDD, ACEKiSing, M4Singer) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Duration Prediction (NTUML2021) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1779.042 MSE · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Duration Prediction (NTUML2021) · MSE · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

53.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Speech Classification (MAESTRO, Librispeech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

17% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sound Effect Detection (RemFx) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Detection (LibriSpeech-TestClean) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Detection (LibriSpeech-TestOther) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Detection (LJSpeech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100 % WER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Children Song Transcript Verification (CSD) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.159 Sacre Bleu · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Lyric Translation (SingSet) · Sacre Bleu · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.1753 MER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Song Lyric Recognition (SingSet) · MER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

9% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Vocal Technique Detection (VocalSet) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

84% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Segment Retrieval (Clotho) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0 IoU · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Segment Retrieval (Clotho) · IoU · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

7.1% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Sound Event Detection (DCASE2016 Task2) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

8.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-channel Sound Event Understanding (STARSS23) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4.3% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Animal Classification (WaveSource-Test) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Bird Sound Detection (Warblrb10k) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Cat Emotion Classification (Cat Sound Classification Dataset V2) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3.3% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Cornell Birdcall Identification · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Animals) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

61% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Beehive States Classification (Beehive States) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

98.33% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Domestic Environment Sound Event Detection (DESED-PublicEval) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1 Event-based F1 · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Domestic Environment Sound EventDetection (DESED-PublicEval) · Event-based F1 · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emergency Traffic Detection (Large-Scale-Audio-dataset) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Exterior And Urban Noises) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Interior And Domestic Sounds) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Natural Soundscapes And Water Sounds) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (UrbanSound8K-Urban Noises) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environment Recognition (ESC50) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4.3% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Environmental Sound Classification (ESC50) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4.17% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Vocal Imitation Classification (Vocal Imitations) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

92.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Spatial Distance Prediction (Spatial LibriSpeech) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.634 MEDAE · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Spatial Distance Prediction (Spatial LibriSpeech) · MEDAE · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

19% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · How Far Are You (3DSpeaker) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sound Position Prediction · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

42% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Emotion Classification (Daily Talk) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.2% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emoji Grounded Speech Emotion Recognition (RAVDESS) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

11.3% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emotion Change Detection (Ravdess) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emotion Recognition (MELD) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13.6% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Emotion Recognition (CREMAD) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13.75% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Emotion Recognition (RAVDESS) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

42.1% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Covid19 Cough Audio Classification (CoughVid) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Human And Non Speech Sounds) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

17.1% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Human Non Speech Sound Recognition (Nonspeech7k, CommonVoice) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

47.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Human Screaming Detection (Environmentdb) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Vocal Sound Recognition (VocalSound) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4.97% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Segment Counting (Librispeech-words) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.007 ACC · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Segment Counting (Librispeech-words) · ACC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

15.214 Abs Diff · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Segment Counting (Librispeech-words) · Abs Diff · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46.88% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phone Segment Counting (VoxAngeles) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.11 ACC · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phone Segment Counting (VoxAngeles) · ACC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3.919 Abs Diff · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phone Segment Counting (VoxAngeles) · Abs Diff · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

110.27 % PER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Phoneme Recognition (LibriSpeech-TestClean) · PER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

111.18 % PER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Recognition (LibriSpeech-TestOther) · PER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3.1% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Consonant Place Of Articulation) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

8% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Manner Of Articulation) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.6% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Phone) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

42.2% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Vowel Frontness) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

39.1% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Vowel Height) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

21.8% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Vowel Roundedness) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

29% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Heteronym Differentiation (HeteronymEn) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

51.9% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Accuracy (speechocean762-Ranking) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

61.88% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Accuracy (speechocean762-scoring) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.037 PCC · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Accuracy (speechocean762-scoring) · PCC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

20.2% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-lingual Pronunciation Similarity (VoxAngeles) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

37% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Accent Classification (Accentdb Extended) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Stress Detection (MIRSD) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

56.25% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Third Tone Sandhi Recognition (NCCU Corpus of Spoken Taiwan Mandarin) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.143 IoU · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Third Tone Sandhi Recognition (NCCU Corpus of Spoken Taiwan Mandarin) · IoU · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50.6% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Fluency (speechocean762-Ranking) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

14.27% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Fluency (speechocean762-Scoring) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.311 PCC · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Fluency (speechocean762-Scoring) · PCC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

52.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Prosodic (speechocean762-Ranking) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

82.1% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Prosodic (speechocean762-Scoring) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

-0.067 PCC · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Prosodic (speechocean762-Scoring) · PCC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

74.9% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Prosody Naturalness (ProsAudit-Lexical) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

75.2% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Prosody Naturalness (ProsAudit-Protosyntax) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

73% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoof Detection (ASVspoof2015) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

66% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoof Detection (ASVspoof2017) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

42% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Deep Fake Voice Recognition (DEEP-VOICE) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

54.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Enhancement Detection (LibriTTS-TestClean, WHAM) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Fraud Robocall Recognition (CallHome) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

47.4% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Fraud Robocall Recognition (Promo) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

38.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Fraud Robocall Recognition (Robocall) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

12.34% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Language Identification (VoxLingua107) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

5.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Language Identification (VoxForge) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

23.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Age Classification (Common Voice) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

51% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Gender Recognition by Voice (Common Voice) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

36.7% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Speaker Count Identification (LibriCount) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

17.69% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Speaker Count Identification (LibriCount) · ACC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

32.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Speaker Detection (VCTK) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Counting (LibriTTS-TestClean) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

36% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Verification (LibriSpeech-TestClean) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

41% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Verification (LibriSpeech-TestOther) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

33.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Verification (VCTK) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Speaker Diarization (Libri2Mix-Test) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

52% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Speaker Verification (SUPERB Hidden Set) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

36.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Gaussian) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

31.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Music) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

48% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Noise) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Speech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

37.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Music) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

54% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Noise) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

48.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Speech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

36% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Gaussian) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

21.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Gaussian) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

14.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Music) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

15.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Noise) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

15.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Speech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (LJSpeech, RirsNoises-LargeRoom) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

48% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (LJSpeech, RirsNoises-MediumRoom) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

44.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (LJSpeech, RirsNoises-SmallRoom) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (VCTK, RirsNoises-LargeRoom) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

47% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (VCTK, RirsNoises-MediumRoom) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

43.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (VCTK, RirsNoises-SmallRoom) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

39.6% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · N-Best Correction (Librispeech-TestOther) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

136.5 % WER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · AAVE Speech Recognition (CORAAL) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

130.18 % MER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-switch Speech Recognition (NTUML2021) · MER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

69.4% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-Switching Speech Recognition (ASCEND) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13.73% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-Switching Speech Recognition (ASCEND) · ACC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

99.37 % WER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-de) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

96.03 % WER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-en) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

103.19 % WER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-es) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

102.25 % WER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-fr) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

102.72 % WER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-it) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

102.05 % WER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-nl) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100.23 % WER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-pl) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

99.71 % WER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-pt) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

639.12 % WER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · PTBR Speech Recognition (Common Voice) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

103.67 % WER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB ASR (LibriSpeech-TestClean) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

111.03 % WER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · ASR (LibriSpeech-TestOther) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

225.28 % WER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Ar) (Common Voice) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

141.68 % WER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Es) (Common Voice) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

137.25 % WER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Spontaneous) (CHIME6-Test) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

435.33 % CER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Zh) (Common Voice) · CER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

108.32 % WER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Target Speaker ASR (AMI-test) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

33.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Speaker Detection (LibriSpeech-TestClean) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

5.2% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Command Recognition (AudioMNIST) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.1 ACC · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Command Recognition (AudioMNIST) · ACC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

41.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Text Matching (LibriSpeech-TestClean) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Text Matching (LibriSpeech-TestOther) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

47.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Text Matching (LJSpeech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

48.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Term Detection (LibriSpeech-TestClean) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

40% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Term Detection (LibriSpeech-TestOther) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

40% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Term Detection (LJSpeech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

27.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Keyword Spotting (Speech Commands V1-Test) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

52% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Query by Example (Quesst14-Eval) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49.8% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Stuttering Detection (SEP28k) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

22.1% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Voice Disorder Classification (VOICED) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.9% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Conversation Matching · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4.9% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Act Classification (SLUE-HVB) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

20.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Act Classification (Daily Talk) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Act Pairing (Daily Talk) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Intent Classification (SLURP) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

7% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Intent Classification (SLURP-Action) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

6% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Intent Classification (SLURP-Intent) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Named Entity Localization (SLUE-VoxPopuli) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

89.13% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Named Entity Recognition (SLUE-VoxPopuli) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.05 IoU · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Named Entity Recognition (SLUE-VoxPopuli) · IoU · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sarcasm Detection (Mustard) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

54% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Semantic Textual Similarity (SpokenSTS) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

28.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Sentiment Analysis (MELD) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

26.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Digit Arithmetic (AudioMNIST) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

95% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Slot Filling (AudioSnips-Test) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.777 Slot Type F1 · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Slot Filling (AudioSnips-Test) · Slot Type F1 · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.669 Slot Value CER · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Slot Filling (AudioSnips-Test) · Slot Value CER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-Switching Semantic Grammar Acceptability Comparison (CSZS-zh-en) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

42.5% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Nonce Word Detection (sWUGGY) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.5327 POS · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · PoS Estimation (LibriTTS) · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.2394 POS · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · PoS Estimation with transcription (LibriTTS) · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

72.8% · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sentence Grammar Acceptability (sBLIMP) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.0566 Sacre Bleu · MU-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Speech Translation (CoVoST2-Test) · Sacre Bleu · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗