Intent Inference on CSID (test)
80LLM Judge AgreementSCRIPTMIND-FT
Evaluation results
| Method | Links | |
|---|---|---|
| SCRIPTMIND-FTbackbone=EEVE-Korean-Instruct-10.8B, fine-tuned=true2026.01 | 80 | |
| chatgpt-4o-latestzero-shot=true2026.01 | 78 | |
| gemini-2.0-flashzero-shot=true2026.01 | 77 | |
| SCRIPTMIND-FTbackbone=EXAONE-3.5-2.4B-Instruct, fine-tuned=true2026.01 | 73 | |
| claude-3-5-haikuzero-shot=true2026.01 | 73 | |
| EXAONE-3.0-7.8B-Instructzero-shot=true2026.01 | 63 | |
| SCRIPTMIND-FTbackbone=Llama-3.2-1B-Instruct, fine-tuned=true2026.01 | 57 | |
| Midm-2.0-Base-Instructzero-shot=true2026.01 | 57 | |
| Llama-3.1-8B-Instructzero-shot=true2026.01 | 54 | |
| SCRIPTMIND-FTbackbone=Llama-3.1-8B-Instruct, fine-tuned=true2026.01 | 54 | |
| SCRIPTMIND-FTbackbone=Midm-2.0-Mini-Instruct, fine-tuned=true2026.01 | 53 | |
| EEVE-Korean-Instruct-10.8Bzero-shot=true2026.01 | 52 | |
| EXAONE-3.5-2.4B-Instructzero-shot=true2026.01 | 51 | |
| SCRIPTMIND-FTbackbone=SOLAR-10.7B-Instruct, fine-tuned=true2026.01 | 50 | |
| SCRIPTMIND-FTbackbone=Midm-2.0-Base-Instruct, fine-tuned=true2026.01 | 50 | |
| SOLAR-10.7B-Instructzero-shot=true2026.01 | 44 | |
| Midm-2.0-Mini-Instructzero-shot=true2026.01 | 29 | |
| Llama-3.2-1B-Instructzero-shot=true2026.01 | 17 |