XiaomiMiMo/verl

XiaomiMiMo/verl

Description: verl/HybridFlow: A Flexible and Efficient RL Post-Training Framework

License: Apache-2.0

Stars: 0

Forks: 0

Open issues: 0

Created: 2026-09-21T05:54:44Z

Pushed: 2026-09-21T01:49:07Z

Default branch: main

Fork: yes

Parent repository: verl-project/verl

Archived: no

README:

!seed logo

verl: Volcano Engine Reinforcement Learning for LLMs

verl is a flexible, efficient and production-ready RL training library for large language models (LLMs).

verl is the open-source version of [HybridFlow: A Flexible and Efficient RLHF Framework](https://arxiv.org/abs/2409.19256v2) paper.

verl is flexible and easy to use with:

• Easy extension of diverse RL algorithms: The hybrid-controller programming model enables flexible representation and efficient execution of complex post-training dataflows. Build RL dataflows such as GRPO, PPO in a few lines of code.

• Seamless integration of existing LLM infra with modular APIs: Decouples computation and data dependencies, enabling seamless integration with existing LLM frameworks, such as FSDP, Megatron-LM, vLLM, SGLang, etc

• Flexible device mapping: Supports various placement of models onto different sets of GPUs for efficient resource utilization and scalability across different cluster sizes.

• Ready integration with popular HuggingFace models

verl is fast with:

• State-of-the-art throughput: SOTA LLM training and inference engine integrations and SOTA RL throughput.

• Efficient actor model resharding with 3D-HybridEngine: Eliminates memory redundancy and significantly reduces communication overhead during transitions between training and generation phases.

News

• [2026/08] verl-vla v0.1.0 is released: a unified VLA post-training framework for human-in-the-loop data collection, supervised fine-tuning, and reinforcement learning across simulators, real robots, and distributed cloud-edge resources, built on top of verl.
• [2026/08] VeRL-Tinker is released: keep the Tinker Cookbook loop you know, and run SFT, RL, and distillation on verl-managed GPU workers you control; read the blog here.
• [2026/08] VeRL-Omni v0.2.0 is released: faster diffusion RL, rebuilt Qwen3-Omni multimodal training (DPO & GSPO), plus LTX-2.3, Qwen-Image-Edit support and more.
• [2026/07] RL-Insight is released: online observability for reinforcement learning training. RL-Insight connects training-side metrics, RL state traces, and service dashboards across distributed rollout and optimization workloads.
• [2026/06] verl-SpeCo is pre-released: a co-training framework for speculative decoding across RL training and inference, keeping draft models aligned during training and reusable for accelerated serving, built on top of verl.
• [2026/05] uni-agent is released: a unified agent framework to build, run, and train LLM agents at scale, built on top of verl.
• [2026/05] VeRL-Omni is pre-released: a unified RL stack for diffusion and omni-modal model post-training built on top of verl. Read the blog post for details.
• [2026/05] verl's zero-mismatch HuggingFace rollout vexact is released: with batch-invariant kernels, shared model definition with FSDP, and out-of-box examples compatible with VeOmni.
• [2026/04] verl's Megatron backend LoRA and router replay support is showcased at PyTorch Conference Europe 2026.
• [2026/03] verl is presented at NVIDIA GTC26: session#1, session#2
• [2026/01] verl has been migrated to the verl-project
• [2026/01] verl first meetup was successfully held in Shanghai on 01/10, hosted by Volcengine and NVIDIA, the slides has been uploaded to verl-data.
• [2026/01] The recipe directory has been migrated to a dedicated repository: verl-recipe and added as a submodule. See https://github.com/verl-project/verl/pull/4795. It can be used as it was after git submodule update --init --recursive recipe. Note that [transfer_queue](verl/experimental/transfer_queue), [fully_async_policy](verl/experimental/fully_async_policy), [one_step_off_policy](verl/experimental/one_step_off_policy) and [vla](verl/experimental/vla) are kept under [verl/experimental](verl/experimental) since they are planned to be merged into the main library. Use them through verl.experimental.{module}.
• [2025/12] Mind Lab successfully used verl and Megatron-bridge to train GRPO Lora for Trillion-parameter model on 64 H800 - See their techblog.
• [2025/10] verl is presented in the PyTorch Conference 2025.
• [2025/08] verl is presented in the PyTorch Expert Exchange Webinar. Slides available.
• [2025/07] The ReTool recipe is fully open sourced. Blog
• [2025/07] The first verl meetup will be held at ICML Vancouver on July 16th! Please join us if you are at ICML! (onsite only)
• [2025/06] verl with Megatron backend enables large MoE models such as [DeepSeek-671B and...