Lightning-AI/pytorch-lightning

Lightning-AI/pytorch-lightning

Description: Pretrain, finetune ANY AI model of ANY size on 1 or 10,000+ GPUs with zero code changes.

Language: Python

License: Apache-2.0

Stars: 31180

Forks: 3736

Open issues: 1003

Created: 2019-03-31T00:45:57Z

Pushed: 2026-06-10T12:56:05Z

Default branch: master

Fork: no

Archived: no

README:

Why PyTorch Lightning?

Training models in plain PyTorch requires writing and maintaining a lot of repetitive engineering code. Handling backpropagation, mixed precision, multi-GPU, and distributed training is error-prone and often reimplemented for every project. PyTorch Lightning organizes PyTorch code to automate this infrastructure while keeping full control over your model logic. You write the science. Lightning handles the engineering, and scales from CPU to multi-node GPUs without changing your core code. PyTorch experts can still opt into [expert-level control](#lightning-fabric-expert-control).

Fun analogy: If PyTorch is Javascript, PyTorch Lightning is ReactJS or NextJS.

Looking for GPUs?

Lightning Cloud is the easiest way to run PyTorch Lightning without managing infrastructure. Start training with one command and get GPUs, autoscaling, monitoring, and a free tier. No cloud setup required.

You can also run PyTorch Lightning on your own hardware or cloud.

Lightning has 2 core packages

[PyTorch Lightning: Train and deploy PyTorch at scale](#why-pytorch-lightning).

[Lightning Fabric: Expert control](#lightning-fabric-expert-control).

Lightning gives you granular control over how much abstraction you want to add over PyTorch.

Quick start

Install Lightning:

pip install lightning

Advanced install options

Install with optional dependencies

pip install lightning['extra']

Conda

conda install lightning -c conda-forge

Install stable version

Install future release from the source

pip install https://github.com/Lightning-AI/lightning/archive/refs/heads/release/stable.zip -U

Install bleeding-edge

Install nightly from the source (no guarantees)

pip install https://github.com/Lightning-AI/lightning/archive/refs/heads/master.zip -U

or from testing PyPI

pip install -iU https://test.pypi.org/simple/ pytorch-lightning

PyTorch Lightning example

Define the training workflow. Here's a toy example (explore real examples):

# main.py
# ! pip install torchvision
import torch, torch.nn as nn, torch.utils.data as data, torchvision as tv, torch.nn.functional as F
import lightning as L

# --------------------------------
# Step 1: Define a LightningModule
# --------------------------------
# A LightningModule (nn.Module subclass) defines a full *system*
# (ie: an LLM, diffusion model, autoencoder, or simple image classifier).

class LitAutoEncoder(L.LightningModule):
def __init__(self):
super().__init__()
self.encoder = nn.Sequential(nn.Linear(28 * 28, 128), nn.ReLU(), nn.Linear(128, 3))
self.decoder = nn.Sequential(nn.Linear(3, 128), nn.ReLU(), nn.Linear(128, 28 * 28))

def forward(self, x):
# in lightning, forward defines the prediction/inference actions
embedding = self.encoder(x)
return embedding

def training_step(self, batch, batch_idx):
# training_step defines the train loop. It is independent of forward
x, _ = batch
x = x.view(x.size(0), -1)
z = self.encoder(x)
x_hat = self.decoder(z)
loss = F.mse_loss(x_hat, x)
self.log("train_loss", loss)
return loss

def configure_optimizers(self):
optimizer = torch.optim.Adam(self.parameters(), lr=1e-3)
return optimizer

# -------------------
# Step 2: Define data
# -------------------
dataset = tv.datasets.MNIST(".", download=True, transform=tv.transforms.ToTensor())
train, val = data.random_split(dataset, [55000, 5000])

# -------------------
# Step 3: Train
# -------------------
autoencoder = LitAutoEncoder()
trainer = L.Trainer()
trainer.fit(autoencoder, data.DataLoader(train), data.DataLoader(val))

Run the model on your terminal

pip install torchvision
python main.py

Convert from PyTorch to PyTorch Lightning

PyTorch Lightning is just organized PyTorch - Lightning disentangles PyTorch code to decouple the science from the engineering.


----

Examples

Explore various types of training possible with PyTorch Lightning. Pretrain and finetune ANY kind of model to perform ANY task like classification, segmentation, summarization and more:

| Task | Description | Run | |------|--------------|-----| | Hello world | Pretrain - Hello world example | | | Image classification | Finetune - ResNet-34 model to classify images of cars | | | Image segmentation | Finetune - ResNet-50 model to segment images | | | Object detection | Finetune - Faster R-CNN model to detect objects | | | Text classification | Finetune - text classifier (BERT model) | | | Text summarization | Finetune - text summarization (Hugging Face transformer model) | | | Audio generation | Finetune - audio generator (transformer model) | | | [LLM…