Multi-turn conversation evaluation on Lost-in-Conversation (test)
95Actions ScoreFull†
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Full†Model=Llama 3.3-70B (Grattafiori et al., 2024)2026.05 | 95 | 91.7 | 72 | 86.2 | |
| Full†Model=GPT-4o-mini (Achiam et al., 2023)2026.05 | 94.1 | 88.1 | 75.9 | 86 | |
| Full†Model=Gemini 2.5 Flash (Team et al., 2023)2026.05 | 88.4 | 90.6 | 97 | 92 | |
| SeDTModel=Llama 3.3-70B (Grattafiori et al., 2024)2026.05 | 85.9 | 71.8 | 61.8 | 73.2 | |
| SeDTModel=Gemini 2.5 Flash (Team et al., 2023)2026.05 | 78.1 | 75.7 | 72 | 75.3 | |
| ShardedModel=Llama 3.3-70B (Grattafiori et al., 2024)2026.05 | 74.7 | 62.9 | 51.6 | 63.1 | |
| SeDTModel=GPT-4o-mini (Achiam et al., 2023)2026.05 | 73 | 74.4 | 58.8 | 68.7 | |
| ShardedModel=GPT-4o-mini (Achiam et al., 2023)2026.05 | 46.7 | 62.9 | 50.4 | 53.3 | |
| ShardedModel=Gemini 2.5 Flash (Team et al., 2023)2026.05 | 40.4 | 65 | 66.4 | 57.3 |