# LLM Training Performance on Alpaca Llama 3.3 70B

Source: [LLM Training Performance on Alpaca Llama 3.3 70B](<https://www.sota2.com/research/sota/llm-training-performance-on-alpaca-llama-3-3-70b>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-21T13:25:41.242Z

Evaluation of the training efficiency and resource utilization for the Alpaca Llama 3.3 70B model, measuring GPU memory consumption and token throughput.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | LLM Training Performance |
| Dataset (including any stated split) | Alpaca Llama 3.3 70B |
| Metric | Memory Usage (GB/GPU) |
| Unit | SIZE |
| Direction | Lower is better |
| Results loaded | 3 of 3 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Memory Usage (GB/GPU)](<https://www.sota2.com/research/sota/llm-training-performance-on-alpaca-llama-3-3-70b.md?metric=Memory%20Usage%20(GB%2FGPU)>): Lower is better; unit: SIZE.

- [Throughput (tokens/s/GPU)](<https://www.sota2.com/research/sota/llm-training-performance-on-alpaca-llama-3-3-70b.md?metric=Throughput%20(tokens%2Fs%2FGPU)>): Higher is better; unit: SCALAR.

## Recorded results

| Method | Memory Usage (GB/GPU) | Result date | Report paper |
| --- | --- | --- | --- |
| torchtune + AC | 74.75 | Not recorded | 6a0e91c7f984739ed6738df1 |
| torchtune + Optim Bwd | 74.89 | Not recorded | 6a0e91c7f984739ed6738df1 |
| torchtune + Compile | 75.22 | Not recorded | 6a0e91c7f984739ed6738df1 |

### torchtune + AC

Memory Usage (GB/GPU): 74.75. Result ID: 6a0f0666fbc373592c07d7f9.

[Reporting paper](<https://www.sota2.com/research/paper/6a0e91c7f984739ed6738df1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a0e91c7f984739ed6738df1.md>)

Recorded evaluation configuration: {"Model":"Llama 3.3 70B","Optimizations":"Activation Checkpointing (AC)","Parallelism":"FSDP2 with tensor, data, and loss parallelism","Hardware":"1x8 H100s","Sequence Length":"2048","Micro-batch size":"2"}. This record does not assert independent reproduction.

### torchtune + Optim Bwd

Memory Usage (GB/GPU): 74.89. Result ID: 6a0f0666fbc373592c07d7fb.

[Reporting paper](<https://www.sota2.com/research/paper/6a0e91c7f984739ed6738df1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a0e91c7f984739ed6738df1.md>)

Recorded evaluation configuration: {"Model":"Llama 3.3 70B","Optimizations":"Optimizer in backward","Parallelism":"FSDP2 with tensor, data, and loss parallelism","Hardware":"1x8 H100s","Sequence Length":"2048","Micro-batch size":"16"}. This record does not assert independent reproduction.

### torchtune + Compile

Memory Usage (GB/GPU): 75.22. Result ID: 6a0f0666fbc373592c07d7fa.

[Reporting paper](<https://www.sota2.com/research/paper/6a0e91c7f984739ed6738df1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a0e91c7f984739ed6738df1.md>)

Recorded evaluation configuration: {"Model":"Llama 3.3 70B","Optimizations":"torch.compile (excluding optimizer step)","Parallelism":"FSDP2 with tensor, data, and loss parallelism","Hardware":"1x8 H100s","Sequence Length":"2048","Micro-batch size":"2"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.