Temporal Leakage Detection on Historical Events (Pre/Post Cutoff)
4.633BPB (Pre-Cutoff)Mr. Chatterbox
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Mr. ChatterboxSize=340M, Cut.=18992026.06 | 4.633 | 4.863 | 1.05 | |
| TYPEWRITERLM ep15Size=8B, Cut.=1913, Epochs=152026.06 | 2.194 | 2.897 | 1.32 | |
| TYPEWRITERLM ep10Size=8B, Cut.=1913, Epochs=102026.06 | 2.164 | 2.864 | 1.32 | |
| TYPEWRITERLM ep5Size=8B, Cut.=1913, Epochs=52026.06 | 1.806 | 2.46 | 1.36 | |
| TimeCapsuleLLMSize=1.2B, Cut.=18752026.06 | 1.65 | 1.96 | 1.19 | |
| GPT-1900 (sft)Size=3.3B, Cut.=19002026.06 | 1.539 | 2.181 | 1.42 | |
| GPT-1900 (base)Size=3.3B, Cut.=19002026.06 | 1.416 | 1.967 | 1.39 | |
| Talkie 1930 (it)Size=13B, Cut.=19302026.06 | 1.32 | 1.87 | 1.42 | |
| TYPEWRITERLM (Self-Instruct)Size=8B, Cut.=19132026.06 | 1.284 | 1.735 | 1.35 | |
| TYPEWRITERLM (base)Size=8B, Cut.=19132026.06 | 1.282 | 1.727 | 1.35 | |
| Talkie 1930 (base)Size=13B, Cut.=19302026.06 | 1.138 | 1.605 | 1.41 | |
| Llama 3.1 8BSize=8B, Cut.=20232026.06 | 0.855 | 1.214 | 1.42 |