Chime-in Reason on MUIR (test)
88.59AccuracyHuman Evaluator
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Human EvaluatorMethod=Baseline, Size=N/A2026.03 | 88.59 | 86.87 | 87.75 | |
| GPT-4oMethod=LLM + Prompt, Size=N/A2026.03 | 87.16 | 84.94 | 72.33 | |
| Qwen-2.5-InstructMethod=SLM + Fine-Tuning, Size=3B2026.03 | 86.28 | 81.02 | 81.25 | |
| Qwen3-MaxMethod=LLM + Prompt, Size=N/A2026.03 | 83.33 | 74.98 | 72.23 | |
| Gemini-2.5-ProMethod=LLM + Prompt, Size=N/A2026.03 | 83.27 | 74.66 | 77.72 | |
| Llama-3.1-InstructMethod=SLM + Fine-Tuning, Size=8B2026.03 | 82.84 | 77.32 | 81 | |
| Qwen-3Method=SLM + Fine-Tuning, Size=8B2026.03 | 81.34 | 67.59 | 64.44 | |
| DeepSeek-V3.2Method=LLM + Prompt, Size=N/A2026.03 | 81.22 | 72.8 | 73.14 | |
| Llama-3.2-InstructMethod=SLM + Fine-Tuning, Size=3B2026.03 | 80.95 | 72.83 | 79.05 | |
| Qwen-2.5-InstructMethod=SLM + Fine-Tuning, Size=7B2026.03 | 80.69 | 62.17 | 74.48 | |
| Phi-4-Mini-InstructMethod=SLM + Fine-Tuning, Size=3.8B2026.03 | 79.85 | 68.7 | 75.97 | |
| Gemma-2-itMethod=SLM + Fine-Tuning, Size=2B2026.03 | 79.41 | 70.27 | 77.83 | |
| Qwen-3Method=SLM + Fine-Tuning, Size=4B2026.03 | 78.59 | 71.82 | 80.62 | |
| Gte-large-en-v1.5Method=Embedding Model + KNN, Size=0.4B2026.03 | 35.6 | 32.61 | 55.79 | |
| Jina-embedding-v3Method=Embedding Model + KNN, Size=0.6B2026.03 | 30.45 | 25.31 | 51.1 | |
| Bge-m3Method=Embedding Model + KNN, Size=0.5B2026.03 | 28.6 | 24.47 | 48.26 | |
| Random GuessMethod=Baseline, Size=N/A2026.03 | 17.21 | 15.52 | 41.2 |