# E-commerce dispute verdict prediction on VerdictBench (test) Source: [E-commerce dispute verdict prediction on VerdictBench (test)]() Collection index: [llms.txt]() Data updated: 2026-05-28T09:52:15.660Z Evaluation of model performance on predicting verdicts for e-commerce disputes using the test split of the VerdictBench dataset. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | E-commerce dispute verdict prediction | | Dataset (including any stated split) | VerdictBench (test) | | Metric | Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 17 of 17 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy](): Higher is better; unit: PERCENT. - [Weighted F1 Score](): Higher is better; unit: PERCENT. - [F1 Score](): Higher is better; unit: PERCENT. - [Recall](): Higher is better; unit: PERCENT. - [Precision](): Higher is better; unit: PERCENT. - [Mean Absolute Error (MAE)](): Lower is better; unit: SCALAR. - [Root Mean Square Error (RMSE)](): Lower is better; unit: SCALAR. ## Recorded results | Method | Accuracy | Result date | Report paper | | --- | --- | --- | --- | | CyberJurors | 72.92 | Not recorded | 6a17cc47dbd7150506f814f6 | | AgentCourt | 66.73 | Not recorded | 6a17cc47dbd7150506f814f6 | | ChatEval | 65.89 | Not recorded | 6a17cc47dbd7150506f814f6 | | Gemini-3-Pro | 63.54 | Not recorded | 6a17cc47dbd7150506f814f6 | | GPT-5.2-Chat | 63.44 | Not recorded | 6a17cc47dbd7150506f814f6 | | DeepSeek-V3 | 60.8 | Not recorded | 6a17cc47dbd7150506f814f6 | | Qwen-Plus | 60.55 | Not recorded | 6a17cc47dbd7150506f814f6 | | Claude-Opus-4.5 | 59.1 | Not recorded | 6a17cc47dbd7150506f814f6 | | Doubao-Seed-1.6 | 56.26 | Not recorded | 6a17cc47dbd7150506f814f6 | | Grok-4 | 54.36 | Not recorded | 6a17cc47dbd7150506f814f6 | | Gemini-2.5-Flash-Lite | 52.92 | Not recorded | 6a17cc47dbd7150506f814f6 | | Llama-2-13B | 50.42 | Not recorded | 6a17cc47dbd7150506f814f6 | | GLM-4 | 50.17 | Not recorded | 6a17cc47dbd7150506f814f6 | | Dolphin3.0-R1-24B | 49.29 | Not recorded | 6a17cc47dbd7150506f814f6 | | Qwen3-VL-235B | 48.43 | Not recorded | 6a17cc47dbd7150506f814f6 | | GPT-5.2 | 48.33 | Not recorded | 6a17cc47dbd7150506f814f6 | | Llama-3.2-90B-Vision | 40.9 | Not recorded | 6a17cc47dbd7150506f814f6 | ### CyberJurors Accuracy: 72.92. Result ID: 6a180f62fbc373592c124492. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Court Simulators","Token":"62,327,703"}. This record does not assert independent reproduction. ### AgentCourt Accuracy: 66.73. Result ID: 6a180f62fbc373592c124491. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Court Simulators","Token":"75,280,342"}. This record does not assert independent reproduction. ### ChatEval Accuracy: 65.89. Result ID: 6a180f62fbc373592c124490. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Court Simulators","Token":"917,151"}. This record does not assert independent reproduction. ### Gemini-3-Pro Accuracy: 63.54. Result ID: 6a180f62fbc373592c124488. [Reporting paper]() Recorded evaluation configuration: {"Type":"Closed MLLM","Token":"4,370,185"}. This record does not assert independent reproduction. ### GPT-5.2-Chat Accuracy: 63.44. Result ID: 6a180f62fbc373592c124484. [Reporting paper]() Recorded evaluation configuration: {"Type":"Closed LLM","Token":"158,189"}. This record does not assert independent reproduction. ### DeepSeek-V3 Accuracy: 60.8. Result ID: 6a180f62fbc373592c124482. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Closed LLM","Token":"114,590"}. This record does not assert independent reproduction. ### Qwen-Plus Accuracy: 60.55. Result ID: 6a180f62fbc373592c124483. [Reporting paper]() Recorded evaluation configuration: {"Type":"Closed LLM","Token":"146,381"}. This record does not assert independent reproduction. ### Claude-Opus-4.5 Accuracy: 59.1. Result ID: 6a180f62fbc373592c12448b. [Reporting paper]() Recorded evaluation configuration: {"Type":"Closed MLLM","Token":"2,698,359"}. This record does not assert independent reproduction. ### Doubao-Seed-1.6 Accuracy: 56.26. Result ID: 6a180f62fbc373592c12448a. [Reporting paper]() Recorded evaluation configuration: {"Type":"Closed MLLM","Token":"4,912,235"}. This record does not assert independent reproduction. ### Grok-4 Accuracy: 54.36. Result ID: 6a180f62fbc373592c12448d. [Reporting paper]() Recorded evaluation configuration: {"Type":"Closed MLLM","Token":"1,188,798"}. This record does not assert independent reproduction. ### Gemini-2.5-Flash-Lite Accuracy: 52.92. Result ID: 6a180f62fbc373592c124489. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Closed MLLM","Token":"839,303"}. This record does not assert independent reproduction. ### Llama-2-13B Accuracy: 50.42. Result ID: 6a180f62fbc373592c124487. [Reporting paper]() Recorded evaluation configuration: {"Type":"Open LLM","Token":"144,899"}. This record does not assert independent reproduction. ### GLM-4 Accuracy: 50.17. Result ID: 6a180f62fbc373592c124485. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Closed LLM","Token":"145,358"}. This record does not assert independent reproduction. ### Dolphin3.0-R1-24B Accuracy: 49.29. Result ID: 6a180f62fbc373592c124486. [Reporting paper]() Recorded evaluation configuration: {"Type":"Open LLM","Token":"145,123"}. This record does not assert independent reproduction. ### Qwen3-VL-235B Accuracy: 48.43. Result ID: 6a180f62fbc373592c12448e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Open MLLM","Token":"2,988,145"}. This record does not assert independent reproduction. ### GPT-5.2 Accuracy: 48.33. Result ID: 6a180f62fbc373592c12448c. [Reporting paper]() Recorded evaluation configuration: {"Type":"Closed MLLM","Token":"1,632,662"}. This record does not assert independent reproduction. ### Llama-3.2-90B-Vision Accuracy: 40.9. Result ID: 6a180f62fbc373592c12448f. [Reporting paper]() Recorded evaluation configuration: {"Type":"Open MLLM","Token":"1,555,677"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.