Text-to-Speech on EmergentTTS (eval)

6.35Overall WERGemini-2.5-Flash-Preview-TTS

Evaluation Results

Method
Gemini-2.5-Flash-Preview-TTS2026.03	6.35	79.1	0.71	97.32	11.8	65.56	18.38	91.25	0.4	74.82	0.5	67.14	—
Gemini-2.5-Flash-Preview-TTS2026.03	6.59	78	0.6	95	12.99	63.03	18.25	89.1	0.36	74.82	0.73	68.03	—
minimax/speech-02-hd2026.03	6.79	40.96	0.57	41.6	14.58	31.96	17.69	33.57	0.27	52.14	0.84	45.53	—
gpt-4o-mini-tts2026.03	7.11	61.98	0.71	67.32	12.07	63.03	21.33	63.39	0.66	54.28	0.84	61.96	—
gpt-4o-mini-audio-preview2026.03	7.18	55.15	0.95	62.9	14.48	64.82	19.04	52.14	0.55	45.71	0.88	50.17	—
Baseline gpt-4o-mini-tts2026.03	7.23	50	0.72	—	13.45	—	20.55	—	0.42	—	1.04	—	—
Fish Audio S12026.03	7.6	36.88	0.75	24.82	15.77	53.75	20.02	37.68	0.72	43.55	0.74	24.64	—
gpt-4o-audio-preview2026.03	7.64	64.9	0.93	67.14	13.75	71.6	20.56	77.32	1.72	49.64	1.26	58.92	—
11Labs eleven multilingual v22026.03	7.64	36.96	0.63	35.35	14.44	36.6	21.51	52.14	0.49	28.21	1.15	32.5	—
gpt-4o-audio-preview2026.03	7.75	77.36	1.82	96.96	13.3	70	21.15	89.46	1.38	62.5	1.16	68.39	—
Gemini-2.5-Pro-Preview-TTS2026.03	7.91	73	0.87	92.14	16.22	63.44	20.87	84.28	0.72	65.71	0.87	60	—
Fish Audio S22026.03	8.15	81.88	0.52	86.61	14.74	63.39	24.58	91.61	0.38	84.41	0.51	83.39	—
ResembleAI Chatterbox2026.03	8.2	27.39	1.18	28.03	17.59	24.64	20.64	17.32	0.65	51.07	0.96	15.89	—
gpt-4o-audio-preview2026.03	8.27	60.53	1.03	56.78	14.72	68.75	23.16	71.6	1.19	48.75	1.25	56.78	—
HumeAI2026.03	8.62	49.12	0.83	71.78	21.05	41.25	19.84	42.14	0.38	42.14	0.93	48.57	—
gpt-4o-mini-audio-preview2026.03	8.86	66.27	9.34	74.1	12.7	73.39	20.92	68.03	0.74	55.53	0.72	60.35	—
KyutAI-TTS2026.03	9.1	24.28	0.83	31.78	16.47	13.39	26.41	24.28	0.61	35.17	1.19	16.78	—
MiniMax/speech-02-hd⋆2026.06	10.02	36.6	—	40.9	—	34.3	—	34.3	—	47.3	—	43.9	16.3
DeepGram Aura-22026.03	10.24	25.21	3.45	17.5	21.41	15.89	23.73	20.89	1.24	43.03	1.36	28.75	—
Gemini-2.5-Flash-TTS⋆2026.06	10.39	70.7	—	95.9	—	58.5	—	91.3	—	63	—	57.9	55.7
dots.tts (SOAR)2026.06	10.45	47.6	—	63.9	—	39.4	—	52.7	—	47	—	65.7	16.4
Kokoro-82M2026.03	10.49	25.46	0.71	18.03	22.17	13.21	28.37	5.89	0.56	43.39	0.65	46.78	—
BASELINE: gpt-4o-mini-tts2026.06	10.61	50	—	—	—	—	—	—	—	—	—	—	—
gpt-4o-mini-tts⋆2026.06	10.76	56.3	—	59.2	—	57.3	—	58.8	—	52.7	—	57.1	52.4
dots.tts (Pretrain)2026.06	10.86	49.2	—	72.7	—	39.5	—	54.7	—	48.4	—	58.4	18
11Labs Multilingual v2⋆2026.06	11.19	33.9	—	30.4	—	35.5	—	45.5	—	39.5	—	35.5	14.5
dots.tts (MF4)2026.06	11.75	47.9	—	59.8	—	36.3	—	55.2	—	50.5	—	64.8	16.7
Gemini-2.5-Pro-TTS⋆2026.06	11.79	69.3	—	86.9	—	58.2	—	82.3	—	61.3	—	61.8	64.8
Higgs Audio V22026.03	11.82	44.64	0.97	69.28	22.26	21.42	31.34	45.53	3.23	45.35	1.29	41.6	—
VoxCPM22026.06	11.84	41.1	—	42.3	—	33.3	—	44.1	—	53.4	—	52.3	18.6
gpt-4o-audio-preview⋆2026.06	11.87	65.2	—	88.8	—	60.2	—	82.1	—	57	—	59.5	40.4
F5TTS2026.03	11.93	19.43	0.7	29.64	23.51	3.21	31.66	19.82	1.62	19.46	2.14	25	—
HumeAI⋆2026.06	12.85	42.7	—	61.6	—	34.6	—	36.9	—	43.2	—	44.6	34.3
Orpheus TTS2026.03	13.63	32.56	1.81	39.06	22.31	14.64	40.94	48.57	1.48	31.07	1.63	29.46	—
F5-TTS2026.06	16.47	15.3	—	26.8	—	1.8	—	21.6	—	14.8	—	23.8	1.4
Qwen3-TTS2026.06	17.32	42.8	—	39.8	—	25.4	—	50.7	—	48.9	—	60.4	30
Qwen 2.5 Omni2026.03	18.34	32.94	2.41	46.6	26.77	14.46	58.44	21.25	0.87	48.92	3.47	33.75	—
Qwen3-TTS2026.06	19.65	42.3	—	60.5	—	17.1	—	62.7	—	56.4	—	43	9.8
Qwen 2.5 Omni2026.03	20.02	30.25	1.22	41.07	26.98	12.5	57.48	20.89	12.77	49.1	1.66	27.67	—
MiniCPM-o2026.03	22.52	19.75	12.36	22.85	33.46	5.89	58.48	21.6	5.21	22.14	3.08	26.25	—