Conversational Question Answering on LoCoMo Open-Domain
20.73F1MemWeaver
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| MemWeaverBackbone=GPT-4o-mini, Avg. Input Tokens=6722026.01 | 20.73 | 15.63 | 5.23 | |
| MemWeaverBackbone=Qwen2.5-1.5B, Avg. Input Tokens=7342026.01 | 17.94 | 15.86 | 4.23 | |
| LoCoMoBackbone=GPT-4o-mini, Avg. Input Tokens=21,6252026.01 | 15.39 | 13.59 | 3.86 | |
| A-MemBackbone=GPT-4o-mini, Avg. Input Tokens=2,7312026.01 | 12.5 | 11.29 | 3.96 | |
| ReadAgentBackbone=Llama3.2-1B, Avg. Input Tokens=6652026.01 | 12.46 | 11.17 | 5.47 | |
| MemWeaverBackbone=Llama3.2-1B, Avg. Input Tokens=10002026.01 | 12.41 | 9.94 | 3.04 | |
| A-MemBackbone=Llama3.2-1B, Avg. Input Tokens=26472026.01 | 12.34 | 7.58 | 2.85 | |
| LoCoMoBackbone=Llama3.2-1B, Avg. Input Tokens=22,3122026.01 | 11.94 | 10.64 | 2.2 | |
| LoCoMoBackbone=Qwen2.5-1.5B, Avg. Input Tokens=22,4242026.01 | 11.58 | 9.74 | 2.37 | |
| A-MemBackbone=Qwen2.5-1.5B, Avg. Input Tokens=2,5742026.01 | 10.83 | 9.21 | 2.43 | |
| MemWeaverBackbone=Llama3.2-3B, Avg. Input Tokens=10002026.01 | 9.64 | 7.77 | 2.11 | |
| LoCoMoBackbone=Llama3.2-3B, Avg. Input Tokens=22,3122026.01 | 7.08 | 5.83 | 0.65 | |
| MemoryBankBackbone=GPT-4o-mini, Avg. Input Tokens=1,5132026.01 | 6.32 | 3.27 | 1.04 | |
| MemoryBankBackbone=Qwen2.5-1.5B, Avg. Input Tokens=1,5552026.01 | 5.8 | 3.4 | 0.82 | |
| MemoryBankBackbone=Llama3.2-1B, Avg. Input Tokens=1,5532026.01 | 5.72 | 3.58 | 0.83 | |
| ReadAgentBackbone=Llama3.2-3B, Avg. Input Tokens=4612026.01 | 5.57 | 5.22 | 5.07 | |
| ReadAgentBackbone=GPT-4o-mini, Avg. Input Tokens=6432026.01 | 5.31 | 5.12 | 0.55 | |
| ReadAgentBackbone=Qwen2.5-1.5B, Avg. Input Tokens=7522026.01 | 5.31 | 12.24 | 0 | |
| A-MemBackbone=Llama3.2-3B, Avg. Input Tokens=25082026.01 | 5.03 | 5.78 | 0.87 | |
| MemoryBankBackbone=Llama3.2-3B, Avg. Input Tokens=1,5532026.01 | 4.52 | 2.09 | 0.83 |