Knowledge Editing on MQuAKE-CF 1.0 (test)
99.9Fact Accuracy (Easy)Learn from stories
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| Learn from storiesBackbone=Llama 3.1, Training Supervision=Learn from stories, Inference Mode=without CoT, Questions=multi-hop2026.02 | 99.9 | 99.8 | 94 | 84.8 | |
| SFTBackbone=Llama 3.1, Training Supervision=SFT, Inference Mode=with CoT, Questions=single-hop2026.02 | 99.6 | 99.5 | 22.6 | 28.2 | |
| SFTBackbone=Llama 3.1, Training Supervision=SFT, Inference Mode=without CoT, Questions=single-hop2026.02 | 99.4 | 99.6 | 14.8 | 24.9 | |
| Learn from storiesBackbone=Llama 3.1, Training Supervision=Learn from stories, Inference Mode=without CoT, Questions=single-hop2026.02 | 99.4 | 99.3 | 68.7 | 52.6 | |
| Learn from storiesBackbone=Llama 3.1, Training Supervision=Learn from stories, Inference Mode=with CoT, Questions=single-hop2026.02 | 98.4 | 98.3 | 76.6 | 70.9 | |
| Learn from factsBackbone=Llama 3.1, Training Supervision=Learn from facts, Inference Mode=without CoT, Questions=multi-hop2026.02 | 97 | 97 | 96.2 | 72 | |
| Learn from factsBackbone=Llama 3.1, Training Supervision=Learn from facts, Inference Mode=without CoT, Questions=single-hop2026.02 | 93.9 | 94.7 | 94 | 32 | |
| Learn from storiesBackbone=Llama 3.1, Training Supervision=Learn from stories, Inference Mode=with CoT, Questions=multi-hop2026.02 | 85.9 | 85.6 | 93 | 78.3 | |
| Learn from storiesBackbone=Llama 3.1, Training Supervision=Learn from stories, Inference Mode=without CoT, Questions=Baseline2026.02 | 65.7 | 64.4 | 76.2 | 18.5 | |
| Learn from factsBackbone=Llama 3.1, Training Supervision=Learn from facts, Inference Mode=without CoT, Questions=Baseline2026.02 | 58 | 58.2 | 71.8 | 14.9 | |
| Learn from storiesBackbone=Llama 3.1, Training Supervision=Learn from stories, Inference Mode=with CoT, Questions=Baseline2026.02 | 46.2 | 45.4 | 84.7 | 21 | |
| Learn from factsBackbone=Llama 3.1, Training Supervision=Learn from facts, Inference Mode=with CoT, Questions=Baseline2026.02 | 45.7 | 48.3 | 76.5 | 17.3 | |
| Learn from factsBackbone=Llama 3.1, Training Supervision=Learn from facts, Inference Mode=with CoT, Questions=single-hop2026.02 | 40.9 | 38.9 | 94.7 | 17.4 | |
| Learn from factsBackbone=Llama 3.1, Training Supervision=Learn from facts, Inference Mode=with CoT, Questions=multi-hop2026.02 | 21.3 | 20.6 | 96.6 | 31.7 |