Conversational Question Answering on LoCoMo Overall
1Avg Rank (F1)MemWeaver
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| MemWeaverBackbone=Llama3.2-3B, Avg. Input Tokens=10002026.01 | 1 | 1.25 | 1.25 | |
| MemWeaverBackbone=Qwen2.5-1.5B, Avg. Input Tokens=7342026.01 | 1 | 1 | 1 | |
| MemWeaverBackbone=GPT-4o-mini, Avg. Input Tokens=6722026.01 | 1.25 | 1 | 1.25 | |
| MemWeaverBackbone=Llama3.2-1B, Avg. Input Tokens=10002026.01 | 1.25 | 1.5 | 1.25 | |
| LoCoMoBackbone=GPT-4o-mini, Avg. Input Tokens=21,6252026.01 | 2 | 2.25 | 2.25 | |
| A-MemBackbone=Llama3.2-1B, Avg. Input Tokens=26472026.01 | 2.25 | 2.5 | 2.25 | |
| A-MemBackbone=Qwen2.5-1.5B, Avg. Input Tokens=2,5742026.01 | 2.25 | 2.5 | 2 | |
| A-MemBackbone=Llama3.2-3B, Avg. Input Tokens=25082026.01 | 2.5 | 2 | 2.25 | |
| A-MemBackbone=GPT-4o-mini, Avg. Input Tokens=2,7312026.01 | 2.75 | 2.75 | 2.5 | |
| LoCoMoBackbone=Llama3.2-3B, Avg. Input Tokens=22,3122026.01 | 2.75 | 2.75 | 4 | |
| LoCoMoBackbone=Qwen2.5-1.5B, Avg. Input Tokens=22,4242026.01 | 2.75 | 3 | 3 | |
| LoCoMoBackbone=Llama3.2-1B, Avg. Input Tokens=22,3122026.01 | 3.25 | 2.75 | 3.25 | |
| ReadAgentBackbone=Llama3.2-1B, Avg. Input Tokens=6652026.01 | 3.25 | 3.25 | 4 | |
| ReadAgentBackbone=GPT-4o-mini, Avg. Input Tokens=6432026.01 | 4.25 | 4 | 4.5 | |
| ReadAgentBackbone=Llama3.2-3B, Avg. Input Tokens=4612026.01 | 4.25 | 4.5 | 2.75 | |
| MemoryBankBackbone=Qwen2.5-1.5B, Avg. Input Tokens=1,5552026.01 | 4.25 | 4.5 | 4 | |
| MemoryBankBackbone=Llama3.2-3B, Avg. Input Tokens=1,5532026.01 | 4.5 | 4.5 | 4.75 | |
| MemoryBankBackbone=GPT-4o-mini, Avg. Input Tokens=1,5132026.01 | 4.75 | 5 | 4.5 | |
| ReadAgentBackbone=Qwen2.5-1.5B, Avg. Input Tokens=7522026.01 | 4.75 | 4 | 5 | |
| MemoryBankBackbone=Llama3.2-1B, Avg. Input Tokens=1,5532026.01 | 5 | 5 | 4.25 |