Fact Accuracy on Synthetic Phonebook Facts Power law Exponent beta = 0
99.9Weighted Fact AccuracyLossHF
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LossHFNumber of Facts in Train Data=1.60M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 99.9 | — | |
| LossHFNumber of Facts in Train Data=1.92M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 99.6 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=1.60M, Selection Variant=Head-Flattened, Model Size=110M parameters2026.04 | 96.6 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=1.60M, Selection Variant=Head, Model Size=110M parameters2026.04 | 96.2 | — | |
| Full DatasetNumber of Facts in Train Data=1.60M, Training Steps=1x steps, Model Size=110M parameters2026.04 | 94.2 | — | |
| Full DatasetNumber of Facts in Train Data=1.60M, Training Steps=8x steps, Model Size=110M parameters2026.04 | 94.2 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=1.60M, Selection Variant=Flattened, Model Size=110M parameters2026.04 | 93.6 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=1.92M, Selection Variant=Flattened, Model Size=110M parameters2026.04 | 93.5 | — | |
| LossHNumber of Facts in Train Data=1.92M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 93.1 | — | |
| Full DatasetNumber of Facts in Train Data=1.92M, Training Steps=1x steps, Model Size=110M parameters2026.04 | 92.9 | — | |
| Full DatasetNumber of Facts in Train Data=1.92M, Training Steps=8x steps, Model Size=110M parameters2026.04 | 92.9 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=1.92M, Selection Variant=Head, Model Size=110M parameters2026.04 | 91.8 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=1.92M, Selection Variant=Head-Flattened, Model Size=110M parameters2026.04 | 91.7 | — | |
| LossHFNumber of Facts in Train Data=2.24M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 91.4 | — | |
| Full DatasetNumber of Facts in Train Data=2.24M, Training Steps=1x steps, Model Size=110M parameters2026.04 | 87.2 | — | |
| Full DatasetNumber of Facts in Train Data=2.24M, Training Steps=8x steps, Model Size=110M parameters2026.04 | 87.2 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=2.24M, Selection Variant=Flattened, Model Size=110M parameters2026.04 | 86.4 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=2.24M, Selection Variant=Head, Model Size=110M parameters2026.04 | 84.9 | — | |
| LossHNumber of Facts in Train Data=2.24M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 84.1 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=2.24M, Selection Variant=Head-Flattened, Model Size=110M parameters2026.04 | 83.4 | — | |
| LossHFNumber of Facts in Train Data=2.56M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 79.8 | — | |
| LossHNumber of Facts in Train Data=2.56M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 74 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=2.56M, Selection Variant=Flattened, Model Size=110M parameters2026.04 | 73.9 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=2.56M, Selection Variant=Head-Flattened, Model Size=110M parameters2026.04 | 73.9 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=2.56M, Selection Variant=Head, Model Size=110M parameters2026.04 | 73.5 | — | |
| Full DatasetNumber of Facts in Train Data=2.56M, Training Steps=8x steps, Model Size=110M parameters2026.04 | 72.6 | — | |
| Full DatasetNumber of Facts in Train Data=2.56M, Training Steps=1x steps, Model Size=110M parameters2026.04 | 67.7 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=5.12M, Selection Variant=Head, Model Size=110M parameters2026.04 | 40.2 | — | |
| LossHNumber of Facts in Train Data=1.60M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 39.6 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=5.12M, Selection Variant=Head-Flattened, Model Size=110M parameters2026.04 | 38 | — | |
| LossHFNumber of Facts in Train Data=5.12M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 37.9 | — | |
| LossHNumber of Facts in Train Data=5.12M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 34.9 | — | |
| LossHFNumber of Facts in Train Data=10.24M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 20.4 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=10.24M, Selection Variant=Head, Model Size=110M parameters2026.04 | 19.2 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=10.24M, Selection Variant=Head-Flattened, Model Size=110M parameters2026.04 | 18.3 | — | |
| LossHNumber of Facts in Train Data=10.24M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 18.3 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=5.12M, Selection Variant=Flattened, Model Size=110M parameters2026.04 | 1.7 | — | |
| Full DatasetNumber of Facts in Train Data=5.12M, Training Steps=1x steps, Model Size=110M parameters2026.04 | 0.3 | — | |
| Full DatasetNumber of Facts in Train Data=5.12M, Training Steps=8x steps, Model Size=110M parameters2026.04 | 0.2 | — | |
| Full DatasetNumber of Facts in Train Data=10.24M, Training Steps=1x steps, Model Size=110M parameters2026.04 | 0 | — | |
| Full DatasetNumber of Facts in Train Data=10.24M, Training Steps=8x steps, Model Size=110M parameters2026.04 | 0 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=10.24M, Selection Variant=Flattened, Model Size=110M parameters2026.04 | 0 | — | |
| Full Dataset (1x steps)Selection Scheme=Full Dataset, Selection Strategy=1x steps, Number of Facts in Train Data=1.60M, Training Steps=800k, Model Size=110M2026.04 | — | 1.53 | |
| Full Dataset (8x steps)Selection Scheme=Full Dataset, Selection Strategy=8x steps, Number of Facts in Train Data=1.60M, Training Steps=6.4m, Model Size=110M2026.04 | — | 1.53 | |
| LossHSelection Scheme=Loss-based Selection, Selection Strategy=LossH, Number of Facts in Train Data=1.60M, Training Steps=800k, Model Size=110M2026.04 | — | 1.55 | |
| LossHFSelection Scheme=Loss-based Selection, Selection Strategy=LossHF, Number of Facts in Train Data=1.60M, Training Steps=800k, Model Size=110M2026.04 | — | 1.6 | |
| LossHFSelection Scheme=Loss-based Selection, Selection Strategy=LossHF, Number of Facts in Train Data=1.92M, Training Steps=800k, Model Size=110M2026.04 | — | 1.91 | |
| LossHFSelection Scheme=Loss-based Selection, Selection Strategy=LossHF, Number of Facts in Train Data=2.24M, Training Steps=800k, Model Size=110M2026.04 | — | 2.05 | |
| LossHFSelection Scheme=Loss-based Selection, Selection Strategy=LossHF, Number of Facts in Train Data=2.56M, Training Steps=800k, Model Size=110M2026.04 | — | 2.04 | |
| LossHFSelection Scheme=Loss-based Selection, Selection Strategy=LossHF, Number of Facts in Train Data=5.12M, Training Steps=800k, Model Size=110M2026.04 | — | 1.94 | |
| LossHFSelection Scheme=Loss-based Selection, Selection Strategy=LossHF, Number of Facts in Train Data=10.24M, Training Steps=800k, Model Size=110M2026.04 | — | 2.11 | |
| Oracle-Aided Selection (Flattened)Selection Scheme=Oracle-Aided Selection, Selection Strategy=Flattened, Number of Facts in Train Data=1.60M, Training Steps=800k, Model Size=110M2026.04 | — | 1.5 | |
| Oracle-Aided Selection (Head-Flattened)Selection Scheme=Oracle-Aided Selection, Selection Strategy=Head-Flattened, Number of Facts in Train Data=1.60M, Training Steps=800k, Model Size=110M2026.04 | — | 1.55 | |
| Oracle-Aided Selection (Head)Selection Scheme=Oracle-Aided Selection, Selection Strategy=Head, Number of Facts in Train Data=1.60M, Training Steps=800k, Model Size=110M2026.04 | — | 1.55 |