Agent Task Performance on Claw-Eval Continuous Mode v1 (test)
63.4Overall PerformanceVanilla
Evaluation results
| Method | Links | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| VanillaMode=Continuous2026.06 | 63.4 | 70.8 | 80.3 | 26.7 | 27.8 | 62.2 | 73.4 | 78.4 | 63.6 | 20 | 41 | 69.6 | 709.845 | 21.981 | 2.622 | 81.52 | |
| AgentSwingMode=Continuous2026.06 | 62.2 | 67.6 | 66.5 | 48.6 | 36.8 | 70 | 63.8 | 90.7 | 31.7 | 22.4 | 41 | 72.8 | 53.776 | 10.027 | 0.907 | 15.63 | |
| SummaryMode=Continuous2026.06 | 61.6 | 63.6 | 74.5 | 35.3 | 20.6 | 55.5 | 70.1 | 87.1 | 66.1 | 69 | 42.6 | 66.9 | 59.772 | 10.143 | 1.001 | 16.59 | |
| LCMMode=Continuous2026.06 | 61.4 | 66.8 | 69 | 38.3 | 29.5 | 63.3 | 74.9 | 66.6 | 67.3 | 20 | 41 | 72.7 | 383.007 | 28.714 | 2.691 | 62.37 | |
| PichayMode=Continuous2026.06 | 61 | 69.5 | 63.8 | 40.3 | 24 | 63.1 | 67 | 94.1 | 52.5 | 21.6 | 41 | 71 | 97.431 | 63.51 | 1.046 | 59.65 | |
| TokenPilotMode=Continuous2026.06 | 60.8 | 58.8 | 61.8 | 52.5 | 32.1 | 64.2 | 57.3 | 89.2 | 65.8 | 76.8 | 45.2 | 70.9 | 21.43 | 9.928 | 0.338 | 10.58 | |
| Keep-Last-NMode=Continuous2026.06 | 60.7 | 65.3 | 74 | 35.5 | 20.8 | 54.1 | 73.6 | 91.9 | 35.7 | 59.5 | 42.4 | 64.7 | 44.812 | 9.106 | 0.78 | 13.7 | |
| LLMLingua-2Mode=Continuous2026.06 | 59 | 58.7 | 71.3 | 34.8 | 30.6 | 61.9 | 65.3 | 77.6 | 64.6 | 20 | 41 | 72.4 | 575.654 | 37.197 | 2.63 | 82.91 | |
| MemoBrainMode=Continuous2026.06 | 57.9 | 65.9 | 55 | 24.9 | 36.7 | 47.8 | 73.5 | 64.2 | 60.6 | 20 | 38.4 | 81.6 | 47.497 | 13.99 | 1.134 | 19.16 | |
| MemOSMode=Continuous2026.06 | 57.7 | 55.9 | 65 | 56.3 | 22.2 | 44.8 | 64.6 | 68.8 | 89 | 20 | 39.6 | 71.5 | 49.742 | 25.432 | 0.293 | 24.12 | |
| SelectiveContextMode=Continuous2026.06 | 56.5 | 58.1 | 71.6 | 21.8 | 21.2 | 54.7 | 74 | 57.7 | 66.4 | 20 | 41 | 72.3 | 437.114 | 48.678 | 2.754 | 81.69 |