# LLM Training Performance on Alpaca Qwen3 32B

Source: [LLM Training Performance on Alpaca Qwen3 32B](<https://www.sota2.com/research/sota/llm-training-performance-on-alpaca-qwen3-32b>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-21T13:25:41.196Z

Evaluation of the memory footprint (GB/GPU) and token throughput (tokens/s/GPU) achieved during the training process of the Alpaca Qwen3 32B model.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | LLM Training Performance |
| Dataset (including any stated split) | Alpaca Qwen3 32B |
| Metric | Memory (GB/GPU) |
| Unit | SIZE |
| Direction | Lower is better |
| Results loaded | 6 of 6 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Memory (GB/GPU)](<https://www.sota2.com/research/sota/llm-training-performance-on-alpaca-qwen3-32b.md?metric=Memory%20(GB%2FGPU)>): Lower is better; unit: SIZE.

- [Throughput (tokens/s/GPU)](<https://www.sota2.com/research/sota/llm-training-performance-on-alpaca-qwen3-32b.md?metric=Throughput%20(tokens%2Fs%2FGPU)>): Higher is better; unit: SCALAR.

## Recorded results

| Method | Memory (GB/GPU) | Result date | Report paper |
| --- | --- | --- | --- |
| torchtune + AC + LCE | 38.43 | Not recorded | 6a0e91c7f984739ed6738df1 |
| Axolotl | 40.9 | Not recorded | 6a0e91c7f984739ed6738df1 |
| torchtune + AC | 42.93 | Not recorded | 6a0e91c7f984739ed6738df1 |
| torchtune + AC + LCE + Compile | 44.17 | Not recorded | 6a0e91c7f984739ed6738df1 |
| torchtune + AC + LCE + Compile + Optim Bwd | 60.41 | Not recorded | 6a0e91c7f984739ed6738df1 |
| torchtune | 67.78 | Not recorded | 6a0e91c7f984739ed6738df1 |

### torchtune + AC + LCE

Memory (GB/GPU): 38.43. Result ID: 6a0f0666fbc373592c07d7f5.

[Reporting paper](<https://www.sota2.com/research/paper/6a0e91c7f984739ed6738df1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a0e91c7f984739ed6738df1.md>)

Recorded evaluation configuration: {"Model":"Qwen3 32B","Optimizations":"AC, Linear Cross Entropy (LCE)","Parallelism":"FSDP2 with tensor, data, and loss parallelism","Hardware":"1x8 H100s","Sequence Length":"2048","Micro-batch size":"2"}. This record does not assert independent reproduction.

### Axolotl

Memory (GB/GPU): 40.9. Result ID: 6a0f0666fbc373592c07d7f8.

[Reporting paper](<https://www.sota2.com/research/paper/6a0e91c7f984739ed6738df1.md>)

Recorded evaluation configuration: {"Model":"Qwen3 32B","Hardware":"1x8 H100s","Sequence Length":"2048","Micro-batch size":"2"}. This record does not assert independent reproduction.

### torchtune + AC

Memory (GB/GPU): 42.93. Result ID: 6a0f0666fbc373592c07d7f4.

[Reporting paper](<https://www.sota2.com/research/paper/6a0e91c7f984739ed6738df1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a0e91c7f984739ed6738df1.md>)

Recorded evaluation configuration: {"Model":"Qwen3 32B","Optimizations":"Activation Checkpointing (AC)","Parallelism":"FSDP2 with tensor, data, and loss parallelism","Hardware":"1x8 H100s","Sequence Length":"2048","Micro-batch size":"2"}. This record does not assert independent reproduction.

### torchtune + AC + LCE + Compile

Memory (GB/GPU): 44.17. Result ID: 6a0f0666fbc373592c07d7f6.

[Reporting paper](<https://www.sota2.com/research/paper/6a0e91c7f984739ed6738df1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a0e91c7f984739ed6738df1.md>)

Recorded evaluation configuration: {"Model":"Qwen3 32B","Optimizations":"AC, LCE, torch.compile","Parallelism":"FSDP2 with tensor, data, and loss parallelism","Hardware":"1x8 H100s","Sequence Length":"2048","Micro-batch size":"2"}. This record does not assert independent reproduction.

### torchtune + AC + LCE + Compile + Optim Bwd

Memory (GB/GPU): 60.41. Result ID: 6a0f0666fbc373592c07d7f7.

[Reporting paper](<https://www.sota2.com/research/paper/6a0e91c7f984739ed6738df1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a0e91c7f984739ed6738df1.md>)

Recorded evaluation configuration: {"Model":"Qwen3 32B","Optimizations":"AC, LCE, torch.compile, Optimizer in backward","Parallelism":"FSDP2 with tensor, data, and loss parallelism","Hardware":"1x8 H100s","Sequence Length":"2048","Micro-batch size":"16"}. This record does not assert independent reproduction.

### torchtune

Memory (GB/GPU): 67.78. Result ID: 6a0f0666fbc373592c07d7f3.

[Reporting paper](<https://www.sota2.com/research/paper/6a0e91c7f984739ed6738df1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a0e91c7f984739ed6738df1.md>)

Recorded evaluation configuration: {"Model":"Qwen3 32B","Optimizations":"None","Parallelism":"FSDP2 with tensor, data, and loss parallelism","Hardware":"1x8 H100s","Sequence Length":"2048","Micro-batch size":"2"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.