E-commerce dispute verdict prediction on VerdictBench (test)
72.92AccuracyCyberJurors
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| CyberJurorsType=Court Simulators, Token=62,327,7032026.05 | 72.92 | 72.58 | 70.37 | 69.99 | 71 | 4.7312 | 6.3724 | |
| AgentCourtType=Court Simulators, Token=75,280,3422026.05 | 66.73 | 66.44 | 63.83 | 63.65 | 64.13 | — | — | |
| ChatEvalType=Court Simulators, Token=917,1512026.05 | 65.89 | 66.45 | 65.25 | 66.78 | 65.69 | — | — | |
| Gemini-3-ProType=Closed MLLM, Token=4,370,1852026.05 | 63.54 | 63.78 | 63.51 | 67.17 | 66.7 | — | — | |
| GPT-5.2-ChatType=Closed LLM, Token=158,1892026.05 | 63.44 | 63.09 | 63.4 | 68.75 | 69.54 | — | — | |
| DeepSeek-V3Type=Closed LLM, Token=114,5902026.05 | 60.8 | 60.42 | 60.75 | 65.93 | 66.62 | — | — | |
| Qwen-PlusType=Closed LLM, Token=146,3812026.05 | 60.55 | 59.7 | 60.37 | 66.58 | 68.26 | — | — | |
| Claude-Opus-4.5Type=Closed MLLM, Token=2,698,3592026.05 | 59.1 | 59.01 | 59.1 | 63.44 | 63.57 | — | — | |
| Doubao-Seed-1.6Type=Closed MLLM, Token=4,912,2352026.05 | 56.26 | 55.67 | 56.17 | 61.39 | 62.09 | — | — | |
| Grok-4Type=Closed MLLM, Token=1,188,7982026.05 | 54.36 | 55.12 | 53.63 | 54.71 | 54.41 | — | — | |
| Gemini-2.5-Flash-LiteType=Closed MLLM, Token=839,3032026.05 | 52.92 | 51.66 | 52.62 | 58.76 | 59.87 | — | — | |
| Llama-2-13BType=Open LLM, Token=144,8992026.05 | 50.42 | 46.79 | 48.98 | 58.54 | 42.24 | — | — | |
| GLM-4Type=Closed LLM, Token=145,3582026.05 | 50.17 | 46.55 | 48.73 | 58.23 | 61.8 | — | — | |
| Dolphin3.0-R1-24BType=Open LLM, Token=145,1232026.05 | 49.29 | 45.68 | 47.9 | 57.47 | 60.65 | — | — | |
| Qwen3-VL-235BType=Open MLLM, Token=2,988,1452026.05 | 48.43 | 47.48 | 48.25 | 53.38 | 53.67 | — | — | |
| GPT-5.2Type=Closed MLLM, Token=1,632,6622026.05 | 48.33 | 49.07 | 47.98 | 49.44 | 49.48 | — | — | |
| Llama-3.2-90B-VisionType=Open MLLM, Token=1,555,6772026.05 | 40.9 | 41.87 | 39.87 | 40.18 | 40.81 | — | — |