Continual Learning on 3-phase text domains (Prose, Python, Math)
-0.007BWTTFGN + LLaMA 3.1 8B (TFGN_LLAMA8B_RETROFIT)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| TFGN + LLaMA 3.1 8B (TFGN_LLAMA8B_RETROFIT)Total params=~9 B, Regime=RF, Phases=32026.05 | -0.007 | 30 | — | |
| TFGN + LLaMA 3.1 8B (TFGN_LLAMA8B_FS)Total params=~9 B, Regime=FS, Phases=32026.05 | -0.095 | — | — | |
| Baseline Std-FT LLaMA 3.1 8B (FS, 500 M tok/phase)Total params=~8 B, Regime=FS, Phases=32026.05 | -0.374 | — | — |