# Legal Judgment Prediction on ECHR (test) Source: [Legal Judgment Prediction on ECHR (test)]() Collection index: [llms.txt]() Data updated: 2026-06-23T07:44:12.024Z Evaluation of model performance on predicting legal judgments using the test split of the ECHR dataset, measured using Accuracy, Coverage, CD-Gain, CD-Cost, and Net-Gain. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Legal Judgment Prediction | | Dataset (including any stated split) | ECHR (test) | | Metric | Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 27 of 27 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy](): Higher is better; unit: PERCENT. - [Coverage](): Higher is better; unit: PERCENT. - [CD-Gain](): Higher is better; unit: PERCENT. - [CD-Cost](): Lower is better; unit: PERCENT. - [Net-Gain](): Higher is better; unit: SCALAR. ## Recorded results | Method | Accuracy | Result date | Report paper | | --- | --- | --- | --- | | FC-Last3-RFN | 82.15 | Not recorded | 6a3a1317206542251d7c6ffe | | Attn-Last3-RFN | 81.5 | Not recorded | 6a3a1317206542251d7c6ffe | | FC-Mid3-RFN | 81.16 | Not recorded | 6a3a1317206542251d7c6ffe | | FC-Mid3-RFN | 79.39 | Not recorded | 6a3a1317206542251d7c6ffe | | Attn-Mid3-RFN | 78.06 | Not recorded | 6a3a1317206542251d7c6ffe | | Attn-Last3-RFN | 75.33 | Not recorded | 6a3a1317206542251d7c6ffe | | Attn-Last3-REV | 75.14 | Not recorded | 6a3a1317206542251d7c6ffe | | Attn-Last3-RFN | 74.26 | Not recorded | 6a3a1317206542251d7c6ffe | | FC-Mid3-RFN | 74.16 | Not recorded | 6a3a1317206542251d7c6ffe | | Attn-Mid3-REV | 73.07 | Not recorded | 6a3a1317206542251d7c6ffe | | Attn-Last3-REV | 73.05 | Not recorded | 6a3a1317206542251d7c6ffe | | FC-Last3-REV | 72.26 | Not recorded | 6a3a1317206542251d7c6ffe | | FC-Mid3-REV | 71.63 | Not recorded | 6a3a1317206542251d7c6ffe | | FC-Last3-RFN | 71.4 | Not recorded | 6a3a1317206542251d7c6ffe | | Attn-Mid3-RFN | 71.4 | Not recorded | 6a3a1317206542251d7c6ffe | | FC-Mid3-REV | 70.14 | Not recorded | 6a3a1317206542251d7c6ffe | | Attn-Last3-REV | 69.44 | Not recorded | 6a3a1317206542251d7c6ffe | | FC-Last3-REV | 69.2 | Not recorded | 6a3a1317206542251d7c6ffe | | Attn-Mid3-REV | 69.2 | Not recorded | 6a3a1317206542251d7c6ffe | | FC-Last3-RFN | 66.16 | Not recorded | 6a3a1317206542251d7c6ffe | | Attn-Mid3-RFN | 65.67 | Not recorded | 6a3a1317206542251d7c6ffe | | FC-Last3-REV | 65.23 | Not recorded | 6a3a1317206542251d7c6ffe | | Attn-Mid3-REV | 65.16 | Not recorded | 6a3a1317206542251d7c6ffe | | LLM-only | 61.27 | Not recorded | 6a3a1317206542251d7c6ffe | | FC-Mid3-REV | 58.2 | Not recorded | 6a3a1317206542251d7c6ffe | | LLM-only | 55.17 | Not recorded | 6a3a1317206542251d7c6ffe | | LLM-only | 54.14 | Not recorded | 6a3a1317206542251d7c6ffe | ### FC-Last3-RFN Accuracy: 82.15. Result ID: 6a3a38b4fbc373592c3427bf. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Mistral-7b","Setting":"Refrain"}. This record does not assert independent reproduction. ### Attn-Last3-RFN Accuracy: 81.5. Result ID: 6a3a38b4fbc373592c3427c3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Mistral-7b","Setting":"Refrain"}. This record does not assert independent reproduction. ### FC-Mid3-RFN Accuracy: 81.16. Result ID: 6a3a38b4fbc373592c3427bd. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Mistral-7b","Setting":"Refrain"}. This record does not assert independent reproduction. ### FC-Mid3-RFN Accuracy: 79.39. Result ID: 6a3a38b4fbc373592c3427b4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-8b","Setting":"Refrain"}. This record does not assert independent reproduction. ### Attn-Mid3-RFN Accuracy: 78.06. Result ID: 6a3a38b4fbc373592c3427c1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Mistral-7b","Setting":"Refrain"}. This record does not assert independent reproduction. ### Attn-Last3-RFN Accuracy: 75.33. Result ID: 6a3a38b4fbc373592c3427cc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-7b","Setting":"Refrain"}. This record does not assert independent reproduction. ### Attn-Last3-REV Accuracy: 75.14. Result ID: 6a3a38b4fbc373592c3427bb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-8b","Setting":"Reverse"}. This record does not assert independent reproduction. ### Attn-Last3-RFN Accuracy: 74.26. Result ID: 6a3a38b4fbc373592c3427ba. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-8b","Setting":"Refrain"}. This record does not assert independent reproduction. ### FC-Mid3-RFN Accuracy: 74.16. Result ID: 6a3a38b4fbc373592c3427c6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-7b","Setting":"Refrain"}. This record does not assert independent reproduction. ### Attn-Mid3-REV Accuracy: 73.07. Result ID: 6a3a38b4fbc373592c3427c2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Mistral-7b","Setting":"Reverse"}. This record does not assert independent reproduction. ### Attn-Last3-REV Accuracy: 73.05. Result ID: 6a3a38b4fbc373592c3427cd. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-7b","Setting":"Reverse"}. This record does not assert independent reproduction. ### FC-Last3-REV Accuracy: 72.26. Result ID: 6a3a38b4fbc373592c3427c0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Mistral-7b","Setting":"Reverse"}. This record does not assert independent reproduction. ### FC-Mid3-REV Accuracy: 71.63. Result ID: 6a3a38b4fbc373592c3427b5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-8b","Setting":"Reverse"}. This record does not assert independent reproduction. ### FC-Last3-RFN Accuracy: 71.4. Result ID: 6a3a38b4fbc373592c3427b6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-8b","Setting":"Refrain"}. This record does not assert independent reproduction. ### Attn-Mid3-RFN Accuracy: 71.4. Result ID: 6a3a38b4fbc373592c3427b8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-8b","Setting":"Refrain"}. This record does not assert independent reproduction. ### FC-Mid3-REV Accuracy: 70.14. Result ID: 6a3a38b4fbc373592c3427be. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Mistral-7b","Setting":"Reverse"}. This record does not assert independent reproduction. ### Attn-Last3-REV Accuracy: 69.44. Result ID: 6a3a38b4fbc373592c3427c4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Mistral-7b","Setting":"Reverse"}. This record does not assert independent reproduction. ### FC-Last3-REV Accuracy: 69.2. Result ID: 6a3a38b4fbc373592c3427b7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-8b","Setting":"Reverse"}. This record does not assert independent reproduction. ### Attn-Mid3-REV Accuracy: 69.2. Result ID: 6a3a38b4fbc373592c3427b9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-8b","Setting":"Reverse"}. This record does not assert independent reproduction. ### FC-Last3-RFN Accuracy: 66.16. Result ID: 6a3a38b4fbc373592c3427c8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-7b","Setting":"Refrain"}. This record does not assert independent reproduction. ### Attn-Mid3-RFN Accuracy: 65.67. Result ID: 6a3a38b4fbc373592c3427ca. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-7b","Setting":"Refrain"}. This record does not assert independent reproduction. ### FC-Last3-REV Accuracy: 65.23. Result ID: 6a3a38b4fbc373592c3427c9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-7b","Setting":"Reverse"}. This record does not assert independent reproduction. ### Attn-Mid3-REV Accuracy: 65.16. Result ID: 6a3a38b4fbc373592c3427cb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-7b","Setting":"Reverse"}. This record does not assert independent reproduction. ### LLM-only Accuracy: 61.27. Result ID: 6a3a38b4fbc373592c3427bc. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Mistral-7b"}. This record does not assert independent reproduction. ### FC-Mid3-REV Accuracy: 58.2. Result ID: 6a3a38b4fbc373592c3427c7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-7b","Setting":"Reverse"}. This record does not assert independent reproduction. ### LLM-only Accuracy: 55.17. Result ID: 6a3a38b4fbc373592c3427b3. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Llama-8b"}. This record does not assert independent reproduction. ### LLM-only Accuracy: 54.14. Result ID: 6a3a38b4fbc373592c3427c5. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen-7b"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.