togethercomputer/torchtune

togethercomputer/torchtune

Description: PyTorch native post-training library

License: BSD-3-Clause

Stars: 0

Forks: 0

Open issues: 0

Created: 2025-06-26T10:43:22Z

Pushed: 2025-09-19T14:50:16Z

Default branch: main

Fork: yes

Parent repository: meta-pytorch/torchtune

Archived: no

README:

torchtune

 !Integration Tests 

[Overview](#overview-) | [Installation](#installation-%EF%B8%8F) | [Get Started](#get-started-) | **Documentation** | [Community](#community-) | [Citing torchtune](#citing-torchtune-) | [License](#license)

📣 Recent updates 📣

• *May 2025*: torchtune has added support for *Qwen3* models! Check out all the configs [here](recipes/configs/qwen3)
• *April 2025*: Llama4 is now available in torchtune! Try out our full and LoRA finetuning configs [here](recipes/configs/llama4)
• *February 2025*: Multi-node training is officially open for business in torchtune! Full finetune on multiple nodes to take advantage of larger batch sizes and models.
• *December 2024*: torchtune now supports Llama 3.3 70B! Try it out by following our installation instructions [here](#installation-%EF%B8%8F), then run any of the configs [here](recipes/configs/llama3_3).
• *November 2024*: torchtune has released v0.4.0 which includes stable support for exciting features like activation offloading and multimodal QLoRA
• *November 2024*: torchtune has added [Gemma2](recipes/configs/gemma2) to its models!
• *October 2024*: torchtune added support for Qwen2.5 models - find the configs [here](recipes/configs/qwen2_5/)
• *September 2024*: torchtune has support for Llama 3.2 11B Vision, Llama 3.2 3B, and Llama 3.2 1B models! Try them out by following our installation instructions [here](#installation-%EF%B8%8F), then run any of the text configs [here](recipes/configs/llama3_2) or vision configs [here](recipes/configs/llama3_2_vision).

Overview 📚

torchtune is a PyTorch library for easily authoring, post-training, and experimenting with LLMs. It provides:

• Hackable training recipes for SFT, knowledge distillation, DPO, PPO, GRPO, and quantization-aware training
• Simple PyTorch implementations of popular LLMs like Llama, Gemma, Mistral, Phi, Qwen, and more
• Best-in-class memory efficiency, performance improvements, and scaling, utilizing the latest PyTorch APIs
• YAML configs for easily configuring training, evaluation, quantization or inference recipes

Post-training recipes

torchtune supports the entire post-training lifecycle. A successful post-trained model will likely utilize several of the below methods.

Supervised Finetuning (SFT)

| Type of Weight Update | 1 Device | >1 Device | >1 Node | |-----------------------|:--------:|:---------:|:-------:| | Full | ✅ | ✅ | ✅ | | LoRA/QLoRA | ✅ | ✅ | ✅ |

Example: `tune run lora_finetune_single_device --config llama3_2/3B_lora_single_device

You can also run e.g. `tune ls lora_finetune_single_device` for a full list of available configs.

Knowledge Distillation (KD)

| Type of Weight Update | 1 Device | >1 Device | >1 Node | |-----------------------|:--------:|:---------:|:-------:| | Full | ❌ | ❌ | ❌ | | LoRA/QLoRA | ✅ | ✅ | ❌ |

Example: `tune run knowledge_distillation_distributed --config qwen2/1.5B_to_0.5B_KD_lora_distributed

You can also run e.g. `tune ls knowledge_distillation_distributed` for a full list of available configs.

Reinforcement Learning / Reinforcement Learning from Human Feedback (RLHF)

| Method | Type of Weight Update | 1 Device | >1 Device | >1 Node | |------------------------------|-----------------------|:--------:|:---------:|:-------:| | DPO | Full | ❌ | ✅ | ❌ | | | LoRA/QLoRA | ✅ | ✅ | ❌ | | PPO | Full | ✅ | ❌ | ❌ | | | LoRA/QLoRA | ❌ | ❌ | ❌ | | GRPO | Full | 🚧 | ✅ | ✅ | | | LoRA/QLoRA | ❌ | ❌ | ❌ |

Example: `tune run lora_dpo_single_device --config llama3_1/8B_dpo_single_device

You can also run e.g. `tune ls full_dpo_distributed` for a full list of available configs.

Quantization-Aware Training (QAT)

| Type of Weight Update | 1 Device | >1 Device | >1 Node | |-----------------------|:--------:|:---------:|:-------:| | Full | ✅ | ✅ | ❌ | | LoRA/QLoRA | ❌ | ✅ | ❌ |

Example: `tune run qat_distributed --config llama3_1/8B_qat_lora

You can also run e.g. `tune ls qat_distributed or tune ls qat_single_device` for a full list of available configs.

The above configs are just examples to get you started. The full list of recipes can be found [here](recipes/). If you'd like to work on one of the gaps you see, please submit a PR! If there's a entirely new post-training method you'd like to see implemented in torchtune, feel free to open an Issue.

Models

For the above recipes, torchtune supports many state-of-the-art models available on the Hugging Face Hub or Kaggle Hub. Some of our supported models:

| Model | Sizes | |-----------------------------------------------|-----------| | Llama4 | Scout (17B x 16E) [[models](torchtune/models/llama4/_model_builders.py), [configs](recipes/configs/llama4/)] | | Llama3.3 | 70B [[models](torchtune/models/llama3_3/_model_builders.py), [configs](recipes/configs/llama3_3/)] | | Llama3.2-Vision-) | 11B, 90B…