Legal Reasoning on LegalBench (Exact Match)
69Exact MatchReConcile
Evaluation Results
| Method | Links | |
|---|---|---|
| ReConcileModels=Qwen2.5-7B-Instruct, finance-Llama3-8B, Saul-7B-Instruct-v12026.05 | 69 | |
| DarkForestModels=Qwen2.5-7B-Instruct, finance-Llama3-8B, Saul-7B-Instruct-v12026.05 | 68 | |
| Self-ConsistencyModels=Qwen2.5-7B-Instruct, finance-Llama3-8B, Saul-7B-Instruct-v12026.05 | 65 | |
| RefineModels=Qwen2.5-7B-Instruct, finance-Llama3-8B, Saul-7B-Instruct-v12026.05 | 60.8 | |
| DebateModels=Qwen2.5-7B-Instruct, finance-Llama3-8B, Saul-7B-Instruct-v12026.05 | 57.6 | |
| Mixture-of-AgentModels=Qwen2.5-7B-Instruct, finance-Llama3-8B, Saul-7B-Instruct-v12026.05 | 53.8 | |
| Graph-of-Agent (Mean)Models=Qwen2.5-7B-Instruct, finance-Llama3-8B, Saul-7B-Instruct-v12026.05 | 46.4 | |
| Graph-of-Agent (Max)Models=Qwen2.5-7B-Instruct, finance-Llama3-8B, Saul-7B-Instruct-v12026.05 | 32.4 |