# torchtune: PyTorch native post-training library

Source: [torchtune: PyTorch native post-training library](<https://www.sota2.com/research/paper/6a0e91c7f984739ed6738df1>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-22T05:00:12.727Z

Authors: Mark Obozov, Maxime Griot, Joseph Cummings, Evan Smothers, Felipe Mello, Rafi Ayub, Philip John Bontrager, Salman Mohammadi, Ariel Kwiatkowski, Nathan Azrak, Mircea Mironenco

Published: 2026-05-20T17:32:08.000Z

## Abstract

Modern LLMs typically require multistage training pipelines to achieve strong downstream performance, with post-training serving as the main interface for adapting open-weight models. We introduce torchtune, a PyTorch-native library designed to streamline the post-training lifecycle of LLMs, enabling efficient fine-tuning, experimentation, and deployment-oriented workflows. Unlike many existing fine-tuning frameworks, which often optimize for ease of use, specialized recipes, or hardware efficiency at the cost of transparency and extensibility, torchtune emphasizes modularity, hackability, and direct access to the underlying PyTorch components. In this paper, we present the design principles behind torchtune, describe how they are reflected in its model builders, training recipes, and distributed training stack, and evaluate the library across representative post-training settings. We compare against popular fine-tuning frameworks, including Axolotl and Unsloth, and show that torchtune provides strong performance and memory efficiency across many settings while remaining flexible enough for rapid research iteration. These results position torchtune as a practical foundation for reproducible LLMs post-training research.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [LLM Training Performance on Alpaca Qwen3 32B](<https://www.sota2.com/research/sota/llm-training-performance-on-alpaca-qwen3-32b.md>): LLM Training Performance; Alpaca Qwen3 32B; Memory (GB/GPU): 38.43.

- [LLM Training Performance on Alpaca Llama 3.3 70B](<https://www.sota2.com/research/sota/llm-training-performance-on-alpaca-llama-3-3-70b.md>): LLM Training Performance; Alpaca Llama 3.3 70B; Memory Usage (GB/GPU): 74.75.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2605.21442>)

[PDF](<https://arxiv.org/pdf/2605.21442.pdf>)