Fact Accuracy on Synthetic Phonebook Facts Power law Exponent beta = 0.5
99.8Weighted Fact AccuracyLossHF
Evaluation results
| Method | Links | |
|---|---|---|
| LossHFNumber of Facts in Train Data=1.60M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 99.8 | |
| LossHFNumber of Facts in Train Data=1.92M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 99.2 | |
| Oracle-Aided SelectionNumber of Facts in Train Data=1.60M, Selection Variant=Flattened, Model Size=110M parameters2026.04 | 97.1 | |
| LossHFNumber of Facts in Train Data=2.24M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 97.1 | |
| Oracle-Aided SelectionNumber of Facts in Train Data=1.60M, Selection Variant=Head-Flattened, Model Size=110M parameters2026.04 | 95.8 | |
| Oracle-Aided SelectionNumber of Facts in Train Data=1.60M, Selection Variant=Head, Model Size=110M parameters2026.04 | 94.6 | |
| LossHNumber of Facts in Train Data=1.60M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 94.4 | |
| Oracle-Aided SelectionNumber of Facts in Train Data=1.92M, Selection Variant=Flattened, Model Size=110M parameters2026.04 | 93.5 | |
| Oracle-Aided SelectionNumber of Facts in Train Data=1.92M, Selection Variant=Head-Flattened, Model Size=110M parameters2026.04 | 93.5 | |
| Full DatasetNumber of Facts in Train Data=1.60M, Training Steps=1x steps, Model Size=110M parameters2026.04 | 93.2 | |
| Full DatasetNumber of Facts in Train Data=1.60M, Training Steps=8x steps, Model Size=110M parameters2026.04 | 93.2 | |
| LossHFNumber of Facts in Train Data=2.56M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 90.5 | |
| Oracle-Aided SelectionNumber of Facts in Train Data=2.24M, Selection Variant=Flattened, Model Size=110M parameters2026.04 | 88.5 | |
| Oracle-Aided SelectionNumber of Facts in Train Data=1.92M, Selection Variant=Head, Model Size=110M parameters2026.04 | 86.3 | |
| Oracle-Aided SelectionNumber of Facts in Train Data=2.24M, Selection Variant=Head-Flattened, Model Size=110M parameters2026.04 | 86.1 | |
| LossHNumber of Facts in Train Data=1.92M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 85.5 | |
| Full DatasetNumber of Facts in Train Data=1.92M, Training Steps=1x steps, Model Size=110M parameters2026.04 | 81.4 | |
| Full DatasetNumber of Facts in Train Data=1.92M, Training Steps=8x steps, Model Size=110M parameters2026.04 | 81.4 | |
| LossHNumber of Facts in Train Data=2.24M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 79.7 | |
| Oracle-Aided SelectionNumber of Facts in Train Data=2.24M, Selection Variant=Head, Model Size=110M parameters2026.04 | 79.6 | |
| Oracle-Aided SelectionNumber of Facts in Train Data=2.56M, Selection Variant=Head-Flattened, Model Size=110M parameters2026.04 | 78.5 | |
| Oracle-Aided SelectionNumber of Facts in Train Data=2.56M, Selection Variant=Head, Model Size=110M parameters2026.04 | 75.5 | |
| Full DatasetNumber of Facts in Train Data=2.24M, Training Steps=8x steps, Model Size=110M parameters2026.04 | 72.7 | |
| LossHNumber of Facts in Train Data=2.56M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 72.4 | |
| Full DatasetNumber of Facts in Train Data=2.24M, Training Steps=1x steps, Model Size=110M parameters2026.04 | 69.2 | |
| Full DatasetNumber of Facts in Train Data=2.56M, Training Steps=1x steps, Model Size=110M parameters2026.04 | 66 | |
| Full DatasetNumber of Facts in Train Data=2.56M, Training Steps=8x steps, Model Size=110M parameters2026.04 | 62.9 | |
| Oracle-Aided SelectionNumber of Facts in Train Data=2.56M, Selection Variant=Flattened, Model Size=110M parameters2026.04 | 60 | |
| Oracle-Aided SelectionNumber of Facts in Train Data=5.12M, Selection Variant=Head-Flattened, Model Size=110M parameters2026.04 | 56.9 | |
| LossHFNumber of Facts in Train Data=5.12M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 56.6 | |
| Oracle-Aided SelectionNumber of Facts in Train Data=5.12M, Selection Variant=Head, Model Size=110M parameters2026.04 | 52.9 | |
| LossHNumber of Facts in Train Data=5.12M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 46 | |
| Oracle-Aided SelectionNumber of Facts in Train Data=5.12M, Selection Variant=Flattened, Model Size=110M parameters2026.04 | 42.1 | |
| Oracle-Aided SelectionNumber of Facts in Train Data=10.24M, Selection Variant=Head-Flattened, Model Size=110M parameters2026.04 | 40.9 | |
| Full DatasetNumber of Facts in Train Data=5.12M, Training Steps=1x steps, Model Size=110M parameters2026.04 | 39.7 | |
| Full DatasetNumber of Facts in Train Data=5.12M, Training Steps=8x steps, Model Size=110M parameters2026.04 | 39.7 | |
| Oracle-Aided SelectionNumber of Facts in Train Data=10.24M, Selection Variant=Head, Model Size=110M parameters2026.04 | 37.3 | |
| LossHNumber of Facts in Train Data=10.24M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 29 | |
| Oracle-Aided SelectionNumber of Facts in Train Data=10.24M, Selection Variant=Flattened, Model Size=110M parameters2026.04 | 28.9 | |
| Full DatasetNumber of Facts in Train Data=10.24M, Training Steps=1x steps, Model Size=110M parameters2026.04 | 27.6 | |
| Full DatasetNumber of Facts in Train Data=10.24M, Training Steps=8x steps, Model Size=110M parameters2026.04 | 27.6 | |
| LossHFNumber of Facts in Train Data=10.24M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 27.2 | |
| LossHFSelection Strategy=LossHF, Number of Facts in Train Data=2.56M, Model Size=110M2026.04 | 2.24 | |
| Oracle-Aided Selection (Head-Flattened)Selection Strategy=Head-Flattened, Number of Facts in Train Data=10.24M, Model Size=110M2026.04 | 1.64 | |
| LossHFSelection Strategy=LossHF, Number of Facts in Train Data=1.60M, Model Size=110M2026.04 | 1.6 |