Fact Accuracy on Synthetic Phonebook Facts Power law Exponent beta = 1.0
99.8Weighted Fact AccuracyLossHF
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LossHFNumber of Facts in Train Data=1.60M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 99.8 | — | |
| LossHFNumber of Facts in Train Data=1.92M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 99.5 | — | |
| LossHFNumber of Facts in Train Data=2.24M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 98 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=1.60M, Selection Variant=Head, Model Size=110M parameters2026.04 | 97.3 | — | |
| Full DatasetNumber of Facts in Train Data=1.60M, Training Steps=8x steps, Model Size=110M parameters2026.04 | 97.2 | — | |
| Full DatasetNumber of Facts in Train Data=1.60M, Training Steps=1x steps, Model Size=110M parameters2026.04 | 97.1 | — | |
| LossHNumber of Facts in Train Data=1.60M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 97 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=1.60M, Selection Variant=Flattened, Model Size=110M parameters2026.04 | 96.9 | — | |
| LossHFNumber of Facts in Train Data=2.56M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 96.7 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=1.92M, Selection Variant=Head-Flattened, Model Size=110M parameters2026.04 | 96.2 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=1.92M, Selection Variant=Head, Model Size=110M parameters2026.04 | 96 | — | |
| LossHNumber of Facts in Train Data=1.92M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 95.5 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=1.60M, Selection Variant=Head-Flattened, Model Size=110M parameters2026.04 | 95.3 | — | |
| Full DatasetNumber of Facts in Train Data=1.92M, Training Steps=1x steps, Model Size=110M parameters2026.04 | 95.2 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=2.24M, Selection Variant=Head, Model Size=110M parameters2026.04 | 95.1 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=2.24M, Selection Variant=Head-Flattened, Model Size=110M parameters2026.04 | 94.8 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=2.56M, Selection Variant=Head, Model Size=110M parameters2026.04 | 94.3 | — | |
| LossHNumber of Facts in Train Data=2.24M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 94.2 | — | |
| Full DatasetNumber of Facts in Train Data=1.92M, Training Steps=8x steps, Model Size=110M parameters2026.04 | 94 | — | |
| Full DatasetNumber of Facts in Train Data=2.24M, Training Steps=1x steps, Model Size=110M parameters2026.04 | 93.8 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=1.92M, Selection Variant=Flattened, Model Size=110M parameters2026.04 | 93.4 | — | |
| LossHNumber of Facts in Train Data=2.56M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 93.3 | — | |
| Full DatasetNumber of Facts in Train Data=2.56M, Training Steps=1x steps, Model Size=110M parameters2026.04 | 92.8 | — | |
| Full DatasetNumber of Facts in Train Data=2.24M, Training Steps=8x steps, Model Size=110M parameters2026.04 | 92.7 | — | |
| Full DatasetNumber of Facts in Train Data=2.56M, Training Steps=8x steps, Model Size=110M parameters2026.04 | 91.6 | — | |
| LossHFNumber of Facts in Train Data=5.12M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 91.6 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=2.56M, Selection Variant=Head-Flattened, Model Size=110M parameters2026.04 | 90.4 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=5.12M, Selection Variant=Head, Model Size=110M parameters2026.04 | 90.2 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=5.12M, Selection Variant=Head-Flattened, Model Size=110M parameters2026.04 | 89.9 | — | |
| LossHNumber of Facts in Train Data=5.12M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 88.9 | — | |
| Full DatasetNumber of Facts in Train Data=5.12M, Training Steps=1x steps, Model Size=110M parameters2026.04 | 88.3 | — | |
| Full DatasetNumber of Facts in Train Data=5.12M, Training Steps=8x steps, Model Size=110M parameters2026.04 | 87.6 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=10.24M, Selection Variant=Head, Model Size=110M parameters2026.04 | 86.5 | — | |
| LossHFNumber of Facts in Train Data=10.24M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 85.8 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=2.24M, Selection Variant=Flattened, Model Size=110M parameters2026.04 | 85.4 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=10.24M, Selection Variant=Head-Flattened, Model Size=110M parameters2026.04 | 85.3 | — | |
| LossHNumber of Facts in Train Data=10.24M, Data Selection Scheme=Loss-based Selection, Model Size=110M parameters2026.04 | 85 | — | |
| Full DatasetNumber of Facts in Train Data=10.24M, Training Steps=1x steps, Model Size=110M parameters2026.04 | 84 | — | |
| Full DatasetNumber of Facts in Train Data=10.24M, Training Steps=8x steps, Model Size=110M parameters2026.04 | 84 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=2.56M, Selection Variant=Flattened, Model Size=110M parameters2026.04 | 68.4 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=10.24M, Selection Variant=Flattened, Model Size=110M parameters2026.04 | 33.3 | — | |
| Oracle-Aided SelectionNumber of Facts in Train Data=5.12M, Selection Variant=Flattened, Model Size=110M parameters2026.04 | 3.4 | — | |
| LossHFSelection Strategy=LossHF, Number of Facts in Train Data=1.60M, Model Size=110M2026.04 | — | 1.58 | |
| LossHFSelection Strategy=LossHF, Number of Facts in Train Data=1.92M, Model Size=110M2026.04 | — | 1.82 | |
| Oracle-Aided Selection (Flattened)Selection Strategy=Flattened, Number of Facts in Train Data=2.24M, Model Size=110M2026.04 | — | 1.95 | |
| Oracle-Aided Selection (Head-Flattened)Selection Strategy=Head-Flattened, Number of Facts in Train Data=10.24M, Model Size=110M2026.04 | — | 1.87 |