Text-to-Music configuration generation on Held-out 200 prompts (test)
0.163CLAP Score (mean)Embed. Lookup
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Embed. LookupMethod=nearest-neighbor retrieval, Embeddings=all-MiniLM-L6-v22026.07 | 0.163 | 100 | 0.3 | 0.79 | 1.2 | |
| Gemini 3 Flash PreviewProvider=Google, Mode=structured output mode2026.07 | 0.158 | 89 | 5.6 | 0.73 | 6.5 | |
| SFT Fine-tuned (270M)Backbone=Gemma 3 (unsloth/gemma-3-270m-it), Parameters=270M, Fine-tuning=LoRA on TRAIN split, Decoding=constrained via Outlines2026.07 | 0.14 | 91 | 99.5 | 0.47 | 100.2 | |
| Random BaselineDescription=uniformly random valid configuration, Prompt=ignored2026.07 | 0.139 | 100 | 0.01 | 0.56 | 0.7 | |
| Claude Opus 4.5Provider=Anthropic, Mode=structured output mode2026.07 | 0.137 | 100 | 11.9 | 0.57 | 12.6 | |
| Base Untrained (270M)Backbone=Gemma 3 (unsloth/gemma-3-270m-it), Parameters=270M, Fine-tuning=none, Decoding=constrained via Outlines grammar-guided2026.07 | 0.117 | 100 | 59.1 | 0.38 | 59.7 |