Legal Research
85.2Accuracy5-shot Qwen + MEMENTO
Evaluation results
| Method | Links | |
|---|---|---|
| 5-shot Qwen + MEMENTOBackbone=Qwen, Evaluation Protocol=In-context learning, Memory Augmentation=MEMENTO2026.05 | 85.2 | |
| 5-shot QwenBackbone=Qwen, Evaluation Protocol=In-context learning, Memory Augmentation=None2026.05 | 84.2 | |
| MEMENTO + GPT-5-miniBackbone=GPT-5-mini, Evaluation Protocol=Learning from web, Memory Augmentation=Full MEMENTO2026.05 | 83.8 | |
| ReAct + GPT-5-mini + MEMENTOBackbone=GPT-5-mini, Evaluation Protocol=Web as a tool (ReAct), Memory Augmentation=MEMENTO2026.05 | 82.5 | |
| QwenBackbone=Qwen, Evaluation Protocol=LLM (only parametric knowledge), Memory Augmentation=None2026.05 | 80.8 | |
| Qwen + MEMENTOBackbone=Qwen, Evaluation Protocol=LLM (only parametric knowledge), Memory Augmentation=MEMENTO2026.05 | 80.8 | |
| MEMENTO + QwenBackbone=Qwen, Evaluation Protocol=Learning from web, Memory Augmentation=Full MEMENTO2026.05 | 80.8 | |
| AET + GPT-5-miniBackbone=GPT-5-mini, Evaluation Protocol=Learning from web, Memory Augmentation=AET-only2026.05 | 78.3 | |
| ReAct + GPT-5-miniBackbone=GPT-5-mini, Evaluation Protocol=Web as a tool (ReAct), Memory Augmentation=None2026.05 | 76.7 | |
| AET + QwenBackbone=Qwen, Evaluation Protocol=Learning from web, Memory Augmentation=AET-only2026.05 | 76.7 | |
| ReAct + Qwen + MEMENTOBackbone=Qwen, Evaluation Protocol=Web as a tool (ReAct), Memory Augmentation=MEMENTO2026.05 | 74.2 | |
| GPT-5-mini + MEMENTOBackbone=GPT-5-mini, Evaluation Protocol=LLM (only parametric knowledge), Memory Augmentation=MEMENTO2026.05 | 71.7 | |
| GPT-5-miniBackbone=GPT-5-mini, Evaluation Protocol=LLM (only parametric knowledge), Memory Augmentation=None2026.05 | 70 | |
| 5-shot GPT-5-miniBackbone=GPT-5-mini, Evaluation Protocol=In-context learning, Memory Augmentation=None2026.05 | 69.2 | |
| 5-shot GPT-5-mini + MEMENTOBackbone=GPT-5-mini, Evaluation Protocol=In-context learning, Memory Augmentation=MEMENTO2026.05 | 67.5 | |
| ReAct + QwenBackbone=Qwen, Evaluation Protocol=Web as a tool (ReAct), Memory Augmentation=None2026.05 | 59.2 |