# Domain-specific Reasoning on LegalBench Source: [Domain-specific Reasoning on LegalBench]() Collection index: [llms.txt]() Data updated: 2026-03-24T15:35:36.141Z Evaluation of model performance on domain-specific reasoning tasks using the LegalBench dataset, measured by Accuracy and Average Accuracy. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Domain-specific Reasoning | | Dataset (including any stated split) | LegalBench | | Metric | Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 33 of 33 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy](): Higher is better; unit: PERCENT. - [Average Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy | Result date | Report paper | | --- | --- | --- | --- | | Reflexion(3 iters) | 85.26 | Not recorded | 69c21a8e0636e5bf4afc4895 | | Reflexion(3 iters) | 81.05 | Not recorded | 69c21a8e0636e5bf4afc4895 | | RM-Regen | 78.95 | Not recorded | 69c21a8e0636e5bf4afc4895 | | RM-Regen | 75.79 | Not recorded | 69c21a8e0636e5bf4afc4895 | | Reflexion(3 iters) | 75.79 | Not recorded | 69c21a8e0636e5bf4afc4895 | | RM-Primed | 67.37 | Not recorded | 69c21a8e0636e5bf4afc4895 | | RM-Regen | 66.32 | Not recorded | 69c21a8e0636e5bf4afc4895 | | RM-Primed (R+) | 64.21 | Not recorded | 69c21a8e0636e5bf4afc4895 | | RM-Primed | 64.21 | Not recorded | 69c21a8e0636e5bf4afc4895 | | Best-of-N (N=3) | 64.21 | Not recorded | 69c21a8e0636e5bf4afc4895 | | RM-Primed (R+) | 62.11 | Not recorded | 69c21a8e0636e5bf4afc4895 | | RM-Regen | 60 | Not recorded | 69c21a8e0636e5bf4afc4895 | | Contrastive CoT | 57.89 | Not recorded | 69c21a8e0636e5bf4afc4895 | | RM-Regen | 56.84 | Not recorded | 69c21a8e0636e5bf4afc4895 | | Contrastive CoT | 55.79 | Not recorded | 69c21a8e0636e5bf4afc4895 | | ST CoT | 55.79 | Not recorded | 69c21a8e0636e5bf4afc4895 | | Contrastive CoT | 53.68 | Not recorded | 69c21a8e0636e5bf4afc4895 | | RM-Regen | 51.58 | Not recorded | 69c21a8e0636e5bf4afc4895 | | ReflectEvo | 50.53 | Not recorded | 69c21a8e0636e5bf4afc4895 | | Few-shot CoT | 48.42 | Not recorded | 69c21a8e0636e5bf4afc4895 | | ReflectEvo | 48.42 | Not recorded | 69c21a8e0636e5bf4afc4895 | | Self-Refine | 46.32 | Not recorded | 69c21a8e0636e5bf4afc4895 | | Best-of-N (N=3) | 45.26 | Not recorded | 69c21a8e0636e5bf4afc4895 | | ST CoT | 45.26 | Not recorded | 69c21a8e0636e5bf4afc4895 | | Few-shot CoT | 44.21 | Not recorded | 69c21a8e0636e5bf4afc4895 | | ProCo | 44.21 | Not recorded | 69c21a8e0636e5bf4afc4895 | | ProCo | 41.05 | Not recorded | 69c21a8e0636e5bf4afc4895 | | Self-Refine | 41.05 | Not recorded | 69c21a8e0636e5bf4afc4895 | | ProCo | 38.95 | Not recorded | 69c21a8e0636e5bf4afc4895 | | Contrastive CoT | 35.75 | Not recorded | 69c21a8e0636e5bf4afc4895 | | Best-of-N (N=3) | 32.63 | Not recorded | 69c21a8e0636e5bf4afc4895 | | Self-Refine | 28.42 | Not recorded | 69c21a8e0636e5bf4afc4895 | | ST CoT | 23.16 | Not recorded | 69c21a8e0636e5bf4afc4895 | ### Reflexion(3 iters) Accuracy: 85.26. Result ID: 69c2adfa47f34ce924abc1e0. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"GPT-3.5","Verification Setting":"Oracle"}. This record does not assert independent reproduction. ### Reflexion(3 iters) Accuracy: 81.05. Result ID: 69c2adfa47f34ce924abc1e3. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Llama3.1-8B","Verification Setting":"Oracle"}. This record does not assert independent reproduction. ### RM-Regen Accuracy: 78.95. Result ID: 69c2adfa47f34ce924abc1ea. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Gemma2-9B","Verification Setting":"Oracle"}. This record does not assert independent reproduction. ### RM-Regen Accuracy: 75.79. Result ID: 69c2adfa47f34ce924abc1e2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-3.5","Verification Setting":"Oracle"}. This record does not assert independent reproduction. ### Reflexion(3 iters) Accuracy: 75.79. Result ID: 69c2adfa47f34ce924abc1e7. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Gemma2-9B","Verification Setting":"Oracle"}. This record does not assert independent reproduction. ### RM-Primed Accuracy: 67.37. Result ID: 69c2adfa47f34ce924abc0af. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-3.5"}. This record does not assert independent reproduction. ### RM-Regen Accuracy: 66.32. Result ID: 69c2adfa47f34ce924abc1e6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama3.1-8B","Verification Setting":"Oracle"}. This record does not assert independent reproduction. ### RM-Primed (R+) Accuracy: 64.21. Result ID: 69c2adfa47f34ce924abc0ae. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-3.5","R+ selection":"only correct entries used"}. This record does not assert independent reproduction. ### RM-Primed Accuracy: 64.21. Result ID: 69c2adfa47f34ce924abc0b4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama3-8B"}. This record does not assert independent reproduction. ### Best-of-N (N=3) Accuracy: 64.21. Result ID: 69c2adfa47f34ce924abc1e5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama3.1-8B","Verification Setting":"Oracle"}. This record does not assert independent reproduction. ### RM-Primed (R+) Accuracy: 62.11. Result ID: 69c2adfa47f34ce924abc0b3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama3-8B","R+ selection":"only correct entries used"}. This record does not assert independent reproduction. ### RM-Regen Accuracy: 60. Result ID: 69c2adfa47f34ce924abc257. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-3.5"}. This record does not assert independent reproduction. ### Contrastive CoT Accuracy: 57.89. Result ID: 69c2adfa47f34ce924abc0b1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama3-8B","Reflection":"without"}. This record does not assert independent reproduction. ### RM-Regen Accuracy: 56.84. Result ID: 69c2adfa47f34ce924abc25b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama3.1-8B"}. This record does not assert independent reproduction. ### Contrastive CoT Accuracy: 55.79. Result ID: 69c2adfa47f34ce924abc0ad. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-3.5","Reflection":"with"}. This record does not assert independent reproduction. ### ST CoT Accuracy: 55.79. Result ID: 69c2adfa47f34ce924abc25a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama3.1-8B","iterations":"3"}. This record does not assert independent reproduction. ### Contrastive CoT Accuracy: 53.68. Result ID: 69c2adfa47f34ce924abc0b2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama3-8B","Reflection":"with"}. This record does not assert independent reproduction. ### RM-Regen Accuracy: 51.58. Result ID: 69c2adfa47f34ce924abc25f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Gemma2-9B"}. This record does not assert independent reproduction. ### ReflectEvo Accuracy: 50.53. Result ID: 69c2adfa47f34ce924abc1e4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama3.1-8B","Verification Setting":"Oracle"}. This record does not assert independent reproduction. ### Few-shot CoT Accuracy: 48.42. Result ID: 69c2adfa47f34ce924abc0b0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama3-8B"}. This record does not assert independent reproduction. ### ReflectEvo Accuracy: 48.42. Result ID: 69c2adfa47f34ce924abc1e8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Gemma2-9B","Verification Setting":"Oracle"}. This record does not assert independent reproduction. ### Self-Refine Accuracy: 46.32. Result ID: 69c2adfa47f34ce924abc258. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama3.1-8B","iterations":"3"}. This record does not assert independent reproduction. ### Best-of-N (N=3) Accuracy: 45.26. Result ID: 69c2adfa47f34ce924abc1e9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Gemma2-9B","Verification Setting":"Oracle"}. This record does not assert independent reproduction. ### ST CoT Accuracy: 45.26. Result ID: 69c2adfa47f34ce924abc25e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Gemma2-9B","iterations":"3"}. This record does not assert independent reproduction. ### Few-shot CoT Accuracy: 44.21. Result ID: 69c2adfa47f34ce924abc0ab. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-3.5"}. This record does not assert independent reproduction. ### ProCo Accuracy: 44.21. Result ID: 69c2adfa47f34ce924abc25d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Gemma2-9B","iterations":"3"}. This record does not assert independent reproduction. ### ProCo Accuracy: 41.05. Result ID: 69c2adfa47f34ce924abc255. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-3.5","iterations":"3"}. This record does not assert independent reproduction. ### Self-Refine Accuracy: 41.05. Result ID: 69c2adfa47f34ce924abc25c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Gemma2-9B","iterations":"3"}. This record does not assert independent reproduction. ### ProCo Accuracy: 38.95. Result ID: 69c2adfa47f34ce924abc259. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama3.1-8B","iterations":"3"}. This record does not assert independent reproduction. ### Contrastive CoT Accuracy: 35.75. Result ID: 69c2adfa47f34ce924abc0ac. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-3.5","Reflection":"without"}. This record does not assert independent reproduction. ### Best-of-N (N=3) Accuracy: 32.63. Result ID: 69c2adfa47f34ce924abc1e1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-3.5","Verification Setting":"Oracle"}. This record does not assert independent reproduction. ### Self-Refine Accuracy: 28.42. Result ID: 69c2adfa47f34ce924abc254. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-3.5","iterations":"3"}. This record does not assert independent reproduction. ### ST CoT Accuracy: 23.16. Result ID: 69c2adfa47f34ce924abc256. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-3.5","iterations":"3"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.