Directed-arm Reclaim on Memory Arithmetic Session-2
100RR (dir)source-first (hand)
Evaluation results
| Method | Links | |
|---|---|---|
| source-first (hand)Answering Model=Llama2026.06 | 100 | |
| source-first (hand)Answering Model=Sonnet2026.06 | 100 | |
| source-first (hand)Answering Model=Opus2026.06 | 100 | |
| source-first-auto (fix)Answering Model=Sonnet2026.06 | 88 | |
| source-first-auto (fix)Answering Model=Opus2026.06 | 86 | |
| LangChain summaryAnswering Model=Opus2026.06 | 56 | |
| LangChain summaryAnswering Model=Sonnet2026.06 | 55 | |
| source-first-auto (fix)Answering Model=Llama2026.06 | 53 | |
| LangChain summaryAnswering Model=Llama2026.06 | 35 | |
| mem0Answering Model=Sonnet2026.06 | 12 | |
| mem0Answering Model=Opus2026.06 | 11 | |
| mem0Answering Model=Llama2026.06 | 9 | |
| naive vector retrievalAnswering Model=Sonnet2026.06 | 7 | |
| naive vector retrievalAnswering Model=Opus2026.06 | 7 | |
| naive vector retrievalAnswering Model=Llama2026.06 | 4 | |
| lossy (hand)Answering Model=Llama2026.06 | 0 | |
| lossy (hand)Answering Model=Sonnet2026.06 | 0 | |
| lossy (hand)Answering Model=Opus2026.06 | 0 |