Acoustic Consistency on SALMon
97.5Speaker ConsistencyLlama-mimi
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Llama-mimiConfig=1.3B, Evaluator=Localized + Normalized Likelihood Evaluation2026.01 | 97.5 | 89 | 98.5 | 79 | 81.5 | 95 | — | — | — | |
| Llama-mimiConfig=1.3B, Evaluator=Normalized Likelihood-Based Evaluation2026.01 | 96.5 | 88.5 | 100 | 73 | 85.5 | 98.5 | — | — | — | |
| Llama-mimiConfig=1.3B, Evaluator=Localized Likelihood-Based Evaluation2026.01 | 95.5 | 95.5 | 100 | 79.5 | 84.5 | 97.5 | — | — | — | |
| Llama-mimiConfig=1.3B, Evaluator=Windowed Likelihood-Based Evaluation2026.01 | 95 | 83.5 | 96.5 | 79.5 | 71 | 88.5 | — | — | — | |
| HumanEvaluator=Human Topline2026.01 | 91.5 | 97.2 | 98.6 | 83.1 | 88.7 | 94.4 | — | — | — | |
| Flow-SLMConfig=1B-ext., Evaluator=Normalized Likelihood-Based Evaluation2026.01 | 88 | 82.5 | 89.5 | 58 | 74 | 85.5 | — | — | — | |
| Flow-SLMConfig=270M, Evaluator=Normalized Likelihood-Based Evaluation2026.01 | 87 | 78 | 84.5 | 57 | 64 | 81.5 | — | — | — | |
| pGSLMEvaluator=Localized + Normalized Likelihood Evaluation2026.01 | 86.5 | 73.5 | 88.5 | 50 | 60 | 67 | — | — | — | |
| Flow-SLMConfig=1B, Evaluator=Normalized Likelihood-Based Evaluation2026.01 | 86 | 78.5 | 87 | 58.5 | 69.5 | 88.5 | — | — | — | |
| Llama-mimiConfig=1.3B, Evaluator=Global-PPL2026.01 | 83.5 | 79.5 | 85 | 73 | 73 | 91.5 | — | — | — | |
| Spirit LMConfig=Expr., Evaluator=Global-PPL2026.01 | 81.5 | 72.5 | 85.5 | 55.5 | 64 | 55.5 | — | — | — | |
| pGSLMEvaluator=Normalized Likelihood-Based Evaluation2026.01 | 79.5 | 63.5 | 88 | 51.5 | 55.5 | 68 | — | — | — | |
| Spirit LMConfig=Expr., Evaluator=Localized Likelihood-Based Evaluation2026.01 | 78 | 79 | 90.5 | 55.5 | 53 | 62 | — | — | — | |
| Flow-SLMConfig=1B, Evaluator=Global-PPL2026.01 | 77 | 65.5 | 79 | 69 | 66 | 69 | — | — | — | |
| pGSLMEvaluator=Localized Likelihood-Based Evaluation2026.01 | 76.5 | 64.5 | 68 | 50 | 57 | 62.5 | — | — | — | |
| TWISTConfig=7B, Evaluator=Normalized Likelihood-Based Evaluation2026.01 | 76 | 58.5 | 72.5 | 58 | 58.5 | 64.5 | — | — | — | |
| Spirit LMConfig=Expr., Evaluator=Localized + Normalized Likelihood Evaluation2026.01 | 76 | 71.5 | 79.5 | 59.5 | 55.5 | 54 | — | — | — | |
| Flow-SLMConfig=1B-ext., Evaluator=Global-PPL2026.01 | 76 | 64.5 | 80.5 | 69.5 | 64.5 | 73.5 | — | — | — | |
| Flow-SLMConfig=1B-ext., Evaluator=Localized Likelihood-Based Evaluation2026.01 | 76 | 79.5 | 81.5 | 65.5 | 69.5 | 83.5 | — | — | — | |
| TWISTConfig=350M, Evaluator=Normalized Likelihood-Based Evaluation2026.01 | 75 | 58 | 72.5 | 57 | 58 | 62 | — | — | — | |
| Flow-SLMConfig=270M, Evaluator=Global-PPL2026.01 | 75 | 61.5 | 76.5 | 66 | 65 | 56.5 | — | — | — | |
| Flow-SLMConfig=1B-ext., Evaluator=Localized + Normalized Likelihood Evaluation2026.01 | 74.5 | 82 | 81 | 54 | 67.5 | 81 | — | — | — | |
| pGSLMEvaluator=Global-PPL2026.01 | 74 | 56 | 69 | 65.5 | 71.5 | 52 | — | — | — | |
| Flow-SLMConfig=1B, Evaluator=Localized Likelihood-Based Evaluation2026.01 | 73.5 | 79.5 | 79 | 64.5 | 68 | 81.5 | — | — | — | |
| TWISTConfig=1.3B, Evaluator=Normalized Likelihood-Based Evaluation2026.01 | 73 | 57 | 74 | 59 | 61.5 | 62.5 | — | — | — | |
| Flow-SLMConfig=1B, Evaluator=Windowed Likelihood-Based Evaluation2026.01 | 72.5 | 71.5 | 81 | 58 | 65.5 | 78.5 | — | — | — | |
| Flow-SLMConfig=270M, Evaluator=Localized Likelihood-Based Evaluation2026.01 | 72 | 75 | 77.5 | 62.5 | 62 | 67.5 | — | — | — | |
| TWISTConfig=7B, Evaluator=Global-PPL2026.01 | 70.5 | 61.5 | 70 | 55.5 | 59.5 | 61.5 | — | — | — | |
| TWISTConfig=350M, Evaluator=Global-PPL2026.01 | 69.5 | 59 | 68 | 54 | 61.5 | 59 | — | — | — | |
| Spirit LMConfig=Expr., Evaluator=Normalized Likelihood-Based Evaluation2026.01 | 69.5 | 70.5 | 78 | 61 | 55.5 | 54.5 | — | — | — | |
| TASTEConfig=emb, Evaluator=Global-PPL2026.01 | 69.5 | 56.5 | 75.5 | 39.5 | 48.5 | 55.5 | — | — | — | |
| TWISTConfig=1.3B, Evaluator=Global-PPL2026.01 | 69 | 61 | 69.5 | 54.5 | 60.5 | 59.5 | — | — | — | |
| Flow-SLMConfig=270M, Evaluator=Windowed Likelihood-Based Evaluation2026.01 | 69 | 65.5 | 72.5 | 58 | 65.5 | 62 | — | — | — | |
| Flow-SLMConfig=1B, Evaluator=Localized + Normalized Likelihood Evaluation2026.01 | 69 | 80.5 | 80 | 53 | 63 | 78.5 | — | — | — | |
| Flow-SLMConfig=270M, Evaluator=Localized + Normalized Likelihood Evaluation2026.01 | 68.5 | 75.5 | 77 | 54.5 | 63 | 78 | — | — | — | |
| pGSLMEvaluator=Windowed Likelihood-Based Evaluation2026.01 | 67.5 | 52 | 55.5 | 58 | 62.5 | 48 | — | — | — | |
| Spirit LMConfig=base, Evaluator=Global-PPL2026.01 | 67.5 | 52.5 | 67 | 53 | 55.5 | 54.5 | — | — | — | |
| Spirit LMConfig=Expr., Evaluator=Windowed Likelihood-Based Evaluation2026.01 | 66.5 | 63.5 | 77.5 | 53 | 53.5 | 56 | — | — | — | |
| TWISTConfig=1.3B, Evaluator=Windowed Likelihood-Based Evaluation2026.01 | 65 | 54 | 70 | 56.5 | 60 | 62.3 | — | — | — | |
| TWISTConfig=1.3B, Evaluator=Localized + Normalized Likelihood Evaluation2026.01 | 65 | 54.5 | 62 | 59.5 | 61 | 58 | — | — | — | |
| TWISTConfig=7B, Evaluator=Localized + Normalized Likelihood Evaluation2026.01 | 64.5 | 55 | 65 | 58 | 61.5 | 67.5 | — | — | — | |
| TWISTConfig=350M, Evaluator=Localized + Normalized Likelihood Evaluation2026.01 | 63.5 | 57.5 | 58 | 57 | 61.5 | 61 | — | — | — | |
| Flow-SLMConfig=1B-ext., Evaluator=Windowed Likelihood-Based Evaluation2026.01 | 63 | 57.5 | 64 | 65 | 54.5 | 57 | — | — | — | |
| LLM-CODEC2026.04 | 63 | — | 65 | 69 | — | 62.5 | 48 | 71.5 | 61.6 | |
| Spirit LMConfig=base, Evaluator=Localized Likelihood-Based Evaluation2026.01 | 62 | 59 | 68 | 53 | 47.5 | 63.5 | — | — | — | |
| TWISTConfig=350M, Evaluator=Windowed Likelihood-Based Evaluation2026.01 | 61.5 | 53.5 | 67 | 56 | 60 | 63.3 | — | — | — | |
| TWISTConfig=7B, Evaluator=Windowed Likelihood-Based Evaluation2026.01 | 61.5 | 56.5 | 63 | 57.5 | 58.5 | 63.8 | — | — | — | |
| TASTEConfig=emb, Evaluator=Windowed Likelihood-Based Evaluation2026.01 | 61.5 | 53.5 | 61 | 45.5 | 44 | 47.5 | — | — | — | |
| TWISTConfig=7B, Evaluator=Localized Likelihood-Based Evaluation2026.01 | 61 | 50.5 | 60 | 59.3 | 52 | 66.3 | — | — | — | |
| Spirit LMConfig=base, Evaluator=Windowed Likelihood-Based Evaluation2026.01 | 60.5 | 57 | 65 | 57.5 | 54.5 | 63.5 | — | — | — | |
| TWISTConfig=350M, Evaluator=Localized Likelihood-Based Evaluation2026.01 | 59.5 | 52 | 63 | 55.8 | 51.5 | 66.8 | — | — | — | |
| TWISTConfig=1.3B, Evaluator=Localized Likelihood-Based Evaluation2026.01 | 58.5 | 50 | 64 | 57.8 | 51.5 | 66.3 | — | — | — | |
| Spirit LMConfig=base, Evaluator=Normalized Likelihood-Based Evaluation2026.01 | 57.5 | 56 | 62.5 | 56.5 | 50 | 58 | — | — | — | |
| Spirit LMConfig=base, Evaluator=Localized + Normalized Likelihood Evaluation2026.01 | 57 | 61 | 60.5 | 53.5 | 48 | 52.5 | — | — | — | |
| TASTEConfig=emb, Evaluator=Localized Likelihood-Based Evaluation2026.01 | 57 | 54.5 | 60.5 | 44 | 43.5 | 49 | — | — | — | |
| GSLMEvaluator=Localized Likelihood-Based Evaluation2026.01 | 56 | 48.5 | 47.5 | 45 | 45.5 | 47 | — | — | — | |
| TASTEConfig=emb, Evaluator=Normalized Likelihood-Based Evaluation2026.01 | 54.5 | 58.5 | 55.5 | 50.5 | 49 | 48.5 | — | — | — | |
| GSLMEvaluator=Normalized Likelihood-Based Evaluation2026.01 | 54 | 51 | 51.5 | 40 | 43.5 | 50.5 | — | — | — | |
| GSLMEvaluator=Windowed Likelihood-Based Evaluation2026.01 | 53.5 | 52 | 54.5 | 48.5 | 52 | 49.5 | — | — | — | |
| TASTEConfig=emb, Evaluator=Localized + Normalized Likelihood Evaluation2026.01 | 52 | 53 | 55 | 44 | 44.5 | 48 | — | — | — | |
| GSLMEvaluator=Global-PPL2026.01 | 50.5 | 52.5 | 53 | 47.5 | 50.5 | 48 | — | — | — | |
| BigCodec2026.04 | 50.5 | — | 49.5 | 53.5 | — | 43.5 | 48 | 48.5 | 49.4 | |
| UniCodec2026.04 | 49 | — | 53 | 45.5 | — | 53 | 47.5 | 46 | 50.1 | |
| AUV2026.04 | 47.5 | — | 52.5 | 53.5 | — | 44 | 45.5 | 49 | 49.4 | |
| WavTok-L2026.04 | 47 | — | 52.5 | 50.5 | — | 37.5 | 51.5 | 51 | 48.3 | |
| GSLMEvaluator=Localized + Normalized Likelihood Evaluation2026.01 | 46 | 54.5 | 52 | 43.5 | 37.5 | 54 | — | — | — |