# E-Commerce Disputes Verdict on CyberJurors 1.0 (test) Source: [E-Commerce Disputes Verdict on CyberJurors 1.0 (test)]() Collection index: [llms.txt]() Data updated: 2026-05-28T09:52:15.628Z Evaluation of model performance on rendering verdicts for E-Commerce disputes using the test split of the CyberJurors 1.0 dataset, measured by Accuracy and P-value. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | E-Commerce Disputes Verdict | | Dataset (including any stated split) | CyberJurors 1.0 (test) | | Metric | Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 17 of 17 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy](): Higher is better; unit: PERCENT. - [P-value](): Lower is better; unit: SCALAR. ## Recorded results | Method | Accuracy | Result date | Report paper | | --- | --- | --- | --- | | CyberJurors | 72.92 | Not recorded | 6a17cc47dbd7150506f814f6 | | AgentCourt | 66.73 | Not recorded | 6a17cc47dbd7150506f814f6 | | ChatEval | 65.89 | Not recorded | 6a17cc47dbd7150506f814f6 | | Gemini-3-Pro | 63.54 | Not recorded | 6a17cc47dbd7150506f814f6 | | GPT-5.2-Chat | 63.44 | Not recorded | 6a17cc47dbd7150506f814f6 | | DeepSeek-V3 | 60.8 | Not recorded | 6a17cc47dbd7150506f814f6 | | Qwen-Plus | 60.55 | Not recorded | 6a17cc47dbd7150506f814f6 | | Claude-Opus-4.5 | 59.1 | Not recorded | 6a17cc47dbd7150506f814f6 | | Doubao-Seed-1.6 | 56.26 | Not recorded | 6a17cc47dbd7150506f814f6 | | Grok-4 | 54.36 | Not recorded | 6a17cc47dbd7150506f814f6 | | Gemini-2.5-Flash-Lite | 52.92 | Not recorded | 6a17cc47dbd7150506f814f6 | | Llama-2-13B | 50.42 | Not recorded | 6a17cc47dbd7150506f814f6 | | GLM-4 | 50.17 | Not recorded | 6a17cc47dbd7150506f814f6 | | Dolphin3.0-R1-24B | 49.29 | Not recorded | 6a17cc47dbd7150506f814f6 | | Qwen3-VL-235B | 48.43 | Not recorded | 6a17cc47dbd7150506f814f6 | | GPT-5.2 | 48.33 | Not recorded | 6a17cc47dbd7150506f814f6 | | Llama-3.2-90B-Vision | 40.9 | Not recorded | 6a17cc47dbd7150506f814f6 | ### CyberJurors Accuracy: 72.92. Result ID: 6a180f62fbc373592c124481. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Court Simulators"}. This record does not assert independent reproduction. ### AgentCourt Accuracy: 66.73. Result ID: 6a180f62fbc373592c124480. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Court Simulators"}. This record does not assert independent reproduction. ### ChatEval Accuracy: 65.89. Result ID: 6a180f62fbc373592c12447f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Court Simulators"}. This record does not assert independent reproduction. ### Gemini-3-Pro Accuracy: 63.54. Result ID: 6a180f62fbc373592c124477. [Reporting paper]() Recorded evaluation configuration: {"Type":"Closed MLLM"}. This record does not assert independent reproduction. ### GPT-5.2-Chat Accuracy: 63.44. Result ID: 6a180f62fbc373592c124473. [Reporting paper]() Recorded evaluation configuration: {"Type":"Closed LLM"}. This record does not assert independent reproduction. ### DeepSeek-V3 Accuracy: 60.8. Result ID: 6a180f62fbc373592c124471. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Closed LLM"}. This record does not assert independent reproduction. ### Qwen-Plus Accuracy: 60.55. Result ID: 6a180f62fbc373592c124472. [Reporting paper]() Recorded evaluation configuration: {"Type":"Closed LLM"}. This record does not assert independent reproduction. ### Claude-Opus-4.5 Accuracy: 59.1. Result ID: 6a180f62fbc373592c12447a. [Reporting paper]() Recorded evaluation configuration: {"Type":"Closed MLLM"}. This record does not assert independent reproduction. ### Doubao-Seed-1.6 Accuracy: 56.26. Result ID: 6a180f62fbc373592c124479. [Reporting paper]() Recorded evaluation configuration: {"Type":"Closed MLLM"}. This record does not assert independent reproduction. ### Grok-4 Accuracy: 54.36. Result ID: 6a180f62fbc373592c12447c. [Reporting paper]() Recorded evaluation configuration: {"Type":"Closed MLLM"}. This record does not assert independent reproduction. ### Gemini-2.5-Flash-Lite Accuracy: 52.92. Result ID: 6a180f62fbc373592c124478. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Closed MLLM"}. This record does not assert independent reproduction. ### Llama-2-13B Accuracy: 50.42. Result ID: 6a180f62fbc373592c124476. [Reporting paper]() Recorded evaluation configuration: {"Type":"Open LLM"}. This record does not assert independent reproduction. ### GLM-4 Accuracy: 50.17. Result ID: 6a180f62fbc373592c124474. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Closed LLM"}. This record does not assert independent reproduction. ### Dolphin3.0-R1-24B Accuracy: 49.29. Result ID: 6a180f62fbc373592c124475. [Reporting paper]() Recorded evaluation configuration: {"Type":"Open LLM"}. This record does not assert independent reproduction. ### Qwen3-VL-235B Accuracy: 48.43. Result ID: 6a180f62fbc373592c12447d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Open MLLM"}. This record does not assert independent reproduction. ### GPT-5.2 Accuracy: 48.33. Result ID: 6a180f62fbc373592c12447b. [Reporting paper]() Recorded evaluation configuration: {"Type":"Closed MLLM"}. This record does not assert independent reproduction. ### Llama-3.2-90B-Vision Accuracy: 40.9. Result ID: 6a180f62fbc373592c12447e. [Reporting paper]() Recorded evaluation configuration: {"Type":"Open MLLM"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.