Question Answering on Consumer QA (Cos. QA)
98AccuracyQwen3-30B-A3B-Instruct
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Qwen3-30B-A3B-InstructEvaluation Protocol=Zero Shot2026.04 | 98 | 97 | 92 | |
| GPT-4oEvaluation Protocol=Zero Shot2026.04 | 98 | 98 | 81 | |
| LegalDrill-1.7BEvaluation Protocol=Distillation, Teacher Model=Qwen3-30B-A3B-Instruct2026.04 | 96 | 96 | 89 | |
| LegalDrill-1.7BEvaluation Protocol=Distillation, Teacher Model=GPT-4o2026.04 | 94 | 94 | 88 | |
| LegalDrill-0.6BEvaluation Protocol=Distillation, Teacher Model=GPT-4o2026.04 | 86 | 84 | 75 | |
| LegalDrill-0.6BEvaluation Protocol=Distillation, Teacher Model=Qwen3-30B-A3B-Instruct2026.04 | 84 | 83 | 77 | |
| DeepSeek ESFT-16BEvaluation Protocol=Zero Shot2026.04 | 81 | 80 | 74 | |
| Qwen3-1.7BEvaluation Protocol=Zero Shot2026.04 | 79 | 58 | 70 | |
| Qwen3-0.6BEvaluation Protocol=Zero Shot2026.04 | 69 | 34 | 66 | |
| Law-LLM-13BEvaluation Protocol=Zero Shot2026.04 | 49 | 38 | 36 | |
| DiscLaw-13BEvaluation Protocol=Zero Shot2026.04 | 24 | 18 | 11 |