Context-aware turn-taking on SPGI
96.86I1 ScoreLLaMA3.1-8b-instruct
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| LLaMA3.1-8b-instructEvaluation Protocol=Zero-shot, System Prompt Frequency=Single2026.03 | 96.86 | 65.32 | 20.75 | 4.58 | 44.36 | 37.2 | 44.76 | |
| LLaMA3.1-8b-instructEvaluation Protocol=Zero-shot, System Prompt Frequency=Repeated twice2026.03 | 96.86 | 65.32 | 20.75 | 4.58 | 44.36 | 37.2 | 44.76 | |
| Qwen3-4b-instructEvaluation Protocol=Zero-shot, System Prompt Frequency=Single2026.03 | 94.94 | 55.07 | 16.99 | 10.34 | 42.25 | 36.81 | 42.6 | |
| Qwen3-4b-instructEvaluation Protocol=Zero-shot, System Prompt Frequency=Repeated twice2026.03 | 94.94 | 55.07 | 16.99 | 10.34 | 42.25 | 36.81 | 42.6 | |
| Qwen3-8bEvaluation Protocol=Zero-shot, System Prompt Frequency=Single2026.03 | 94.94 | 43.7 | 33.7 | 11.28 | 42.46 | 39.31 | 42.73 | |
| Qwen3-8bEvaluation Protocol=Zero-shot, System Prompt Frequency=Repeated twice2026.03 | 94.94 | 43.7 | 33.7 | 11.28 | 42.46 | 39.31 | 42.73 | |
| Qwen2.5-7bEvaluation Protocol=Zero-shot, System Prompt Frequency=Single2026.03 | 89.28 | 67.95 | 25.72 | 16.3 | 48.15 | 43.68 | 48.48 | |
| Qwen2.5-7bEvaluation Protocol=Zero-shot, System Prompt Frequency=Repeated twice2026.03 | 89.28 | 67.95 | 25.72 | 16.3 | 48.15 | 43.68 | 48.48 | |
| gpt-5.2Evaluation Protocol=Zero-shot, System Prompt Frequency=Single2026.03 | 85.3 | 31.54 | 75.48 | 49.45 | 56.96 | 56.93 | 56.94 | |
| gpt-5.2Evaluation Protocol=Zero-shot, System Prompt Frequency=Repeated twice2026.03 | 85.3 | 31.54 | 75.48 | 49.45 | 56.96 | 56.93 | 56.94 | |
| Mistral-7b-instructEvaluation Protocol=Zero-shot, System Prompt Frequency=Single2026.03 | 84.6 | 72.03 | 24.03 | 19.52 | 49.01 | 44.8 | 49.33 | |
| Mistral-7b-instructEvaluation Protocol=Zero-shot, System Prompt Frequency=Repeated twice2026.03 | 84.6 | 72.03 | 24.03 | 19.52 | 49.01 | 44.8 | 49.33 | |
| gemini-3.1-proEvaluation Protocol=Zero-shot, System Prompt Frequency=Single2026.03 | 79.35 | 30.31 | 91.32 | 69.07 | 64.57 | 63.93 | 64.45 | |
| gemini-3.1-proEvaluation Protocol=Zero-shot, System Prompt Frequency=Repeated twice2026.03 | 79.35 | 30.31 | 91.32 | 69.07 | 64.57 | 63.93 | 64.45 | |
| gpt-oss-20bEvaluation Protocol=Zero-shot, System Prompt Frequency=Single2026.03 | 68.63 | 22.84 | 70.18 | 48.35 | 49.62 | 49.35 | 49.54 | |
| gpt-oss-20bEvaluation Protocol=Zero-shot, System Prompt Frequency=Repeated twice2026.03 | 68.63 | 22.84 | 70.18 | 48.35 | 49.62 | 49.35 | 49.54 |