Open-domain long-horizon evaluation on FRAMES (test)
23.91AccuracyHiMPO
Evaluation Results
| Method | Links | |
|---|---|---|
| HiMPOContext budget=8k ctx., Compression strategy=naive_recency2026.06 | 23.91 | |
| HiMPOContext budget=4k ctx., Compression strategy=naive_recency2026.06 | 23.67 | |
| HiMPOContext budget=full ctx.2026.06 | 23.67 | |
| HiMPOContext budget=16k ctx., Compression strategy=mem_aware2026.06 | 23.54 | |
| HiMPOContext budget=8k ctx., Compression strategy=mem_aware2026.06 | 23.3 | |
| HiMPOContext budget=4k ctx., Compression strategy=mem_aware2026.06 | 23.18 | |
| HiMPOContext budget=16k ctx., Compression strategy=naive_recency2026.06 | 23.18 | |
| SUPOContext budget=8k ctx., Compression strategy=naive_recency2026.06 | 21.6 | |
| SUPOContext budget=4k ctx., Compression strategy=naive_recency2026.06 | 21.36 | |
| SUPOContext budget=16k ctx., Compression strategy=mem_aware2026.06 | 21.36 | |
| MEMPOContext budget=8k ctx., Compression strategy=naive_recency2026.06 | 21.24 | |
| SUPOContext budget=full ctx.2026.06 | 21.24 | |
| MEMPOContext budget=4k ctx., Compression strategy=naive_recency2026.06 | 21 | |
| SUPOContext budget=16k ctx., Compression strategy=naive_recency2026.06 | 21 | |
| SFT onlyContext budget=8k ctx., Compression strategy=naive_recency2026.06 | 20.87 | |
| MEMPOContext budget=16k ctx., Compression strategy=mem_aware2026.06 | 20.87 | |
| MEMPOContext budget=full ctx.2026.06 | 20.87 | |
| SUPOContext budget=8k ctx., Compression strategy=mem_aware2026.06 | 20.87 | |
| SUPOContext budget=4k ctx., Compression strategy=mem_aware2026.06 | 20.75 | |
| ReActContext budget=8k ctx., Compression strategy=naive_recency2026.06 | 20.63 | |
| SFT onlyContext budget=4k ctx., Compression strategy=naive_recency2026.06 | 20.63 | |
| MEMPOContext budget=16k ctx., Compression strategy=naive_recency2026.06 | 20.63 | |
| SFT onlyContext budget=16k ctx., Compression strategy=mem_aware2026.06 | 20.51 | |
| SFT onlyContext budget=full ctx.2026.06 | 20.51 | |
| MEMPOContext budget=8k ctx., Compression strategy=mem_aware2026.06 | 20.51 | |
| MEMPOContext budget=4k ctx., Compression strategy=mem_aware2026.06 | 20.39 | |
| SFT onlyContext budget=16k ctx., Compression strategy=naive_recency2026.06 | 20.27 | |
| SFT onlyContext budget=8k ctx., Compression strategy=mem_aware2026.06 | 20.15 | |
| SFT onlyContext budget=4k ctx., Compression strategy=mem_aware2026.06 | 20.02 | |
| ReActContext budget=16k ctx., Compression strategy=mem_aware2026.06 | 18.57 | |
| ReActContext budget=4k ctx., Compression strategy=mem_aware2026.06 | 17.84 | |
| ReActContext budget=4k ctx., Compression strategy=naive_recency2026.06 | 17.23 | |
| ReActContext budget=full ctx.2026.06 | 17.11 | |
| ReActContext budget=16k ctx., Compression strategy=naive_recency2026.06 | 16.87 | |
| ReActContext budget=8k ctx., Compression strategy=mem_aware2026.06 | 16.02 |