# LLM Training Performance on Qwen models on RTX 3090 (24GB VRAM) 2.5 (train)

Source: [LLM Training Performance on Qwen models on RTX 3090 (24GB VRAM) 2.5 (train)](<https://www.sota2.com/research/sota/llm-training-performance-on-qwen-models-on-rtx-3090-24gb-vram-2-5-train>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-08T07:50:18.304Z

Evaluation of the training performance metrics (TFLOPS, token throughput, and GPU/CPU memory utilization) when training Qwen 2.5 models on an RTX 3090 GPU configuration.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | LLM Training Performance |
| Dataset (including any stated split) | Qwen models on RTX 3090 (24GB VRAM) 2.5 (train) |
| Metric | TFLOPS |
| Unit | SCALAR |
| Direction | Higher is better |
| Results loaded | 5 of 5 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [TFLOPS](<https://www.sota2.com/research/sota/llm-training-performance-on-qwen-models-on-rtx-3090-24gb-vram-2-5-train.md?metric=TFLOPS>): Higher is better; unit: SCALAR.

- [GPU Memory (GB)](<https://www.sota2.com/research/sota/llm-training-performance-on-qwen-models-on-rtx-3090-24gb-vram-2-5-train.md?metric=GPU%20Memory%20(GB)>): Lower is better; unit: SIZE.

- [CPU Memory (GB)](<https://www.sota2.com/research/sota/llm-training-performance-on-qwen-models-on-rtx-3090-24gb-vram-2-5-train.md?metric=CPU%20Memory%20(GB)>): Lower is better; unit: SIZE.

- [Throughput (tokens/s)](<https://www.sota2.com/research/sota/llm-training-performance-on-qwen-models-on-rtx-3090-24gb-vram-2-5-train.md?metric=Throughput%20(tokens%2Fs)>): Higher is better; unit: SCALAR.

## Recorded results

| Method | TFLOPS | Result date | Report paper |
| --- | --- | --- | --- |
| MegaTrain | 35.09 | Not recorded | 69d5e1247d0172140ec75b40 |
| MegaTrain | 33.18 | Not recorded | 69d5e1247d0172140ec75b40 |
| MegaTrain | 30.19 | Not recorded | 69d5e1247d0172140ec75b40 |
| ZeRO-3 Offload | 27.49 | Not recorded | 69d5e1247d0172140ec75b40 |
| ZeRO-3 Offload | 23.91 | Not recorded | 69d5e1247d0172140ec75b40 |

### MegaTrain

TFLOPS: 35.09. Result ID: 69d608b047f34ce924be578e.

[Reporting paper](<https://www.sota2.com/research/paper/69d5e1247d0172140ec75b40.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69d5e1247d0172140ec75b40.md>)

[Implementation](<https://github.com/DLYuanGod/MegaTrain>)

Recorded evaluation configuration: {"Model":"Qwen2.5-7B","Max BS":"5"}. This record does not assert independent reproduction.

### MegaTrain

TFLOPS: 33.18. Result ID: 69d608b047f34ce924be578c.

[Reporting paper](<https://www.sota2.com/research/paper/69d5e1247d0172140ec75b40.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69d5e1247d0172140ec75b40.md>)

[Implementation](<https://github.com/DLYuanGod/MegaTrain>)

Recorded evaluation configuration: {"Model":"Qwen2.5-3B","Max BS":"7"}. This record does not assert independent reproduction.

### MegaTrain

TFLOPS: 30.19. Result ID: 69d608b047f34ce924be5790.

[Reporting paper](<https://www.sota2.com/research/paper/69d5e1247d0172140ec75b40.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69d5e1247d0172140ec75b40.md>)

[Implementation](<https://github.com/DLYuanGod/MegaTrain>)

Recorded evaluation configuration: {"Model":"Qwen2.5-14B","Max BS":"3"}. This record does not assert independent reproduction.

### ZeRO-3 Offload

TFLOPS: 27.49. Result ID: 69d608b047f34ce924be578f.

[Reporting paper](<https://www.sota2.com/research/paper/69d5e1247d0172140ec75b40.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa678f1e209131cea17034.md>)

Recorded evaluation configuration: {"Model":"Qwen2.5-7B","Max BS":"1"}. This record does not assert independent reproduction.

### ZeRO-3 Offload

TFLOPS: 23.91. Result ID: 69d608b047f34ce924be578d.

[Reporting paper](<https://www.sota2.com/research/paper/69d5e1247d0172140ec75b40.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa678f1e209131cea17034.md>)

Recorded evaluation configuration: {"Model":"Qwen2.5-3B","Max BS":"1"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.