Chime-in Timing on MUIR (test)
86.42AccuracyHuman Evaluator
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Human EvaluatorMethod=Baseline, Size=N/A2026.03 | 86.42 | 89.13 | 87.75 | |
| Qwen-3Method=SLM + Fine-Tuning, Size=4B2026.03 | 83.4 | 88.67 | 80.62 | |
| Llama-3.1-InstructMethod=SLM + Fine-Tuning, Size=8B2026.03 | 78.47 | 85.35 | 81 | |
| Llama-3.2-InstructMethod=SLM + Fine-Tuning, Size=3B2026.03 | 77.8 | 84.6 | 79.05 | |
| Gemma-2-itMethod=SLM + Fine-Tuning, Size=2B2026.03 | 77.68 | 83.95 | 77.83 | |
| Qwen-2.5-InstructMethod=SLM + Fine-Tuning, Size=3B2026.03 | 75.36 | 82.32 | 81.25 | |
| Gte-large-en-v1.5Method=Embedding Model + KNN, Size=0.4B2026.03 | 74.28 | 80.65 | 55.79 | |
| Phi-4-Mini-InstructMethod=SLM + Fine-Tuning, Size=3.8B2026.03 | 74.07 | 81.25 | 75.97 | |
| Gemini-2.5-ProMethod=LLM + Prompt, Size=N/A2026.03 | 73.66 | 79.29 | 77.72 | |
| Qwen-2.5-InstructMethod=SLM + Fine-Tuning, Size=7B2026.03 | 73.24 | 81.81 | 74.48 | |
| Jina-embedding-v3Method=Embedding Model + KNN, Size=0.6B2026.03 | 71.6 | 77.03 | 51.1 | |
| DeepSeek-V3.2Method=LLM + Prompt, Size=N/A2026.03 | 67.28 | 71.25 | 73.14 | |
| Bge-m3Method=Embedding Model + KNN, Size=0.5B2026.03 | 66.46 | 73.52 | 48.26 | |
| Qwen3-MaxMethod=LLM + Prompt, Size=N/A2026.03 | 63.58 | 67.04 | 72.23 | |
| Random GuessMethod=Baseline, Size=N/A2026.03 | 59.26 | 72.8 | 41.2 | |
| GPT-4oMethod=LLM + Prompt, Size=N/A2026.03 | 58.02 | 59.2 | 72.33 | |
| Qwen-3Method=SLM + Fine-Tuning, Size=8B2026.03 | 53.8 | 55.03 | 64.44 |