Knowledge Persistence and Factual Retrieval on loop-drift protocol (test)
97.5Short Fact ScoreRouted
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| RoutedModel=TinyLlama, Writes=2.5, L2=21.22026.06 | 97.5 | 63.4 | 77.3 | 75 | |
| RAGModel=TinyLlama, Writes=0.0, L2=0.02026.06 | 97.3 | 66 | 39.6 | 39.4 | |
| RAGModel=GPT-2, Writes=0.0, L2=0.02026.06 | 95.6 | 73.3 | 39.8 | 39.4 | |
| RoutedModel=GPT-2, Writes=2.4, L2=29.32026.06 | 93.5 | 49.2 | 92.7 | 92.5 | |
| Naive-LoRAModel=GPT-2, Writes=200.0, L2=119.32026.06 | 73.3 | 33 | 64.2 | 65 | |
| Naive-LoRAModel=TinyLlama, Writes=200.0, L2=67.52026.06 | 68.5 | 29.4 | 62.3 | 57.5 | |
| EVAFModel=TinyLlama, Writes=2.6, L2=21.52026.06 | 48.3 | 56.5 | 83.3 | 81.2 | |
| EVAFModel=GPT-2, Writes=2.4, L2=29.02026.06 | 46.3 | 55.8 | 90.4 | 90 |