Knowledge Retention on RWKU Neighbor Set
64.64FB ScoreDirectQA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| DirectQAFinetuning Strategy=GA+GD, Backbone=Llama-2-7b-chat2026.04 | 64.64 | 69.53 | 67.09 | |
| DirectQAFinetuning Strategy=NPO, Backbone=Llama-2-7b-chat2026.04 | 63.64 | 72.13 | 67.88 | |
| DirectQAFinetuning Strategy=GA, Backbone=Llama-2-7b-chat2026.04 | 63.53 | 69.79 | 66.66 | |
| BeforeBackbone=Llama-2-7b-chat2026.04 | 63.06 | 74.84 | 68.95 | |
| MAGEFinetuning Strategy=GA, Backbone=Llama-2-7b-chat2026.04 | 62.63 | 65.01 | 63.82 | |
| DirectQAFinetuning Strategy=GA+KL, Backbone=Llama-2-7b-chat2026.04 | 62.39 | 70.64 | 66.51 | |
| ELUDeFinetuning Strategy=GA+KL, Backbone=Llama-2-7b-chat2026.04 | 61.48 | 64.93 | 63.2 | |
| ELUDeFinetuning Strategy=GA, Backbone=Llama-2-7b-chat2026.04 | 61.35 | 69.44 | 65.4 | |
| RWKUFinetuning Strategy=NPO, Backbone=Llama-2-7b-chat2026.04 | 61.03 | 69.25 | 65.14 | |
| MAGEFinetuning Strategy=GA+KL, Backbone=Llama-2-7b-chat2026.04 | 60.19 | 62.95 | 61.57 | |
| MAGEFinetuning Strategy=GA+GD, Backbone=Llama-2-7b-chat2026.04 | 58.82 | 57.88 | 58.35 | |
| ELUDeFinetuning Strategy=GA+GD, Backbone=Llama-2-7b-chat2026.04 | 57.5 | 58.64 | 58.07 | |
| RWKUFinetuning Strategy=GA+GD, Backbone=Llama-2-7b-chat2026.04 | 56.91 | 72.14 | 64.53 | |
| RWKUFinetuning Strategy=GA+KL, Backbone=Llama-2-7b-chat2026.04 | 56.5 | 72.22 | 64.36 | |
| RWKUFinetuning Strategy=GA, Backbone=Llama-2-7b-chat2026.04 | 56 | 71.53 | 63.76 | |
| ELUDeFinetuning Strategy=NPO, Backbone=Llama-2-7b-chat2026.04 | 48.29 | 42.39 | 45.34 | |
| MAGEFinetuning Strategy=NPO, Backbone=Llama-2-7b-chat2026.04 | 42.79 | 38 | 40.4 |