# LLM Training Optimization on Llama 3.2 3B Source: [LLM Training Optimization on Llama 3.2 3B]() Collection index: [llms.txt]() Data updated: 2026-02-26T06:07:38.011Z Evaluation of techniques aimed at optimizing the training process for the Llama 3.2 3B model, specifically measuring the percentage reduction in training time and energy consumption. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | LLM Training Optimization | | Dataset (including any stated split) | Llama 3.2 3B | | Metric | Training Time Reduction (%) | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 20 of 20 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Training Time Reduction (%)](): Higher is better; unit: PERCENT. - [Energy Reduction (%)](): Higher is better; unit: PERCENT. - [Energy Reduction (Iso-Time) (%)](): Higher is better; unit: PERCENT. - [Time Reduction (Iso-Energy) (%)](): Higher is better; unit: PERCENT. ## Recorded results | Method | Training Time Reduction (%) | Result date | Report paper | | --- | --- | --- | --- | | Kareus | 12.3 | Not recorded | 6978467ca2b28e761d87c477 | | Nanobatching + Perseus | 8.5 | Not recorded | 6978467ca2b28e761d87c477 | | Kareus | 8 | Not recorded | 6978467ca2b28e761d87c477 | | Kareus | 6.2 | Not recorded | 6978467ca2b28e761d87c477 | | Kareus | 5.2 | Not recorded | 6978467ca2b28e761d87c477 | | Nanobatching + Perseus | 2.4 | Not recorded | 6978467ca2b28e761d87c477 | | Nanobatching + Perseus | 1.5 | Not recorded | 6978467ca2b28e761d87c477 | | Megatron-LM + Perseus | 0.2 | Not recorded | 6978467ca2b28e761d87c477 | | Nanobatching + Perseus | 0 | Not recorded | 6978467ca2b28e761d87c477 | | Megatron-LM + Perseus | -0.3 | Not recorded | 6978467ca2b28e761d87c477 | | Megatron-LM + Perseus | -0.7 | Not recorded | 6978467ca2b28e761d87c477 | | Megatron-LM + Perseus | -1.4 | Not recorded | 6978467ca2b28e761d87c477 | ### Kareus Training Time Reduction (%): 12.3. Result ID: 69991afb5698665ec4818d1e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Parallelism":"TP8","µBatch Size":"8","Sequence Length":"4K"}. This record does not assert independent reproduction. ### Nanobatching + Perseus Training Time Reduction (%): 8.5. Result ID: 69991afb5698665ec4818d1d. [Reporting paper]() Recorded evaluation configuration: {"Parallelism":"TP8","µBatch Size":"8","Sequence Length":"4K"}. This record does not assert independent reproduction. ### Kareus Training Time Reduction (%): 8. Result ID: 69991afb5698665ec4818d27. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Parallelism":"CP2TP4","µBatch Size":"16","Sequence Length":"4K"}. This record does not assert independent reproduction. ### Kareus Training Time Reduction (%): 6.2. Result ID: 69991afb5698665ec4818d24. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Parallelism":"CP2TP4","µBatch Size":"8","Sequence Length":"8K"}. This record does not assert independent reproduction. ### Kareus Training Time Reduction (%): 5.2. Result ID: 69991afb5698665ec4818d21. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Parallelism":"CP2TP4","µBatch Size":"8","Sequence Length":"4K"}. This record does not assert independent reproduction. ### Nanobatching + Perseus Training Time Reduction (%): 2.4. Result ID: 69991afb5698665ec4818d26. [Reporting paper]() Recorded evaluation configuration: {"Parallelism":"CP2TP4","µBatch Size":"16","Sequence Length":"4K"}. This record does not assert independent reproduction. ### Nanobatching + Perseus Training Time Reduction (%): 1.5. Result ID: 69991afb5698665ec4818d23. [Reporting paper]() Recorded evaluation configuration: {"Parallelism":"CP2TP4","µBatch Size":"8","Sequence Length":"8K"}. This record does not assert independent reproduction. ### Megatron-LM + Perseus Training Time Reduction (%): 0.2. Result ID: 69991afb5698665ec4818d1f. [Reporting paper]() Recorded evaluation configuration: {"Parallelism":"CP2TP4","µBatch Size":"8","Sequence Length":"4K"}. This record does not assert independent reproduction. ### Nanobatching + Perseus Training Time Reduction (%): 0. Result ID: 69991afb5698665ec4818d20. [Reporting paper]() Recorded evaluation configuration: {"Parallelism":"CP2TP4","µBatch Size":"8","Sequence Length":"4K"}. This record does not assert independent reproduction. ### Megatron-LM + Perseus Training Time Reduction (%): -0.3. Result ID: 69991afb5698665ec4818d1c. [Reporting paper]() Recorded evaluation configuration: {"Parallelism":"TP8","µBatch Size":"8","Sequence Length":"4K"}. This record does not assert independent reproduction. ### Megatron-LM + Perseus Training Time Reduction (%): -0.7. Result ID: 69991afb5698665ec4818d25. [Reporting paper]() Recorded evaluation configuration: {"Parallelism":"CP2TP4","µBatch Size":"16","Sequence Length":"4K"}. This record does not assert independent reproduction. ### Megatron-LM + Perseus Training Time Reduction (%): -1.4. Result ID: 69991afb5698665ec4818d22. [Reporting paper]() Recorded evaluation configuration: {"Parallelism":"CP2TP4","µBatch Size":"8","Sequence Length":"8K"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.