novitalabs/pegaflow v0.23.7

v0.23.7

Repository: novitalabs/pegaflow

Tag: v0.23.7

Published: 2026-08-11T03:35:37Z

Prerelease: no

Release notes:

Highlights

• Added vLLM hybrid KV cache model support, including aligned FullAttention + Mamba layouts and hybrid MLA handling.
• Added host-local endpoint shards for cross-host tensor parallel deployments. Each PegaFlow server can now serve the contiguous TP ranks on its host while preserving a shared logical cache namespace.

Compatibility

• The load RPC changed incompatibly to carry grouped layers and per-group destinations. Upgrade PegaFlow servers and vLLM connectors together.
• Cross-host endpoint shards currently support equal, contiguous TP-only layouts. Pipeline, decode-context, and prefill-context parallel combinations are rejected.
• Published wheels cover Python 3.10-3.14 on x86_64 for CUDA 12 and CUDA 13. aarch64 systems currently need a source build.

Validation

• Qwen3.5-4B hybrid-model correctness E2E: 5 tests passed.
• Kimi K3 on B300 x 8: cold and warm outputs matched at cached boundaries; measured median speedup was 2.42x.
• Split local-server E2E: 33.81 req/s cold and 42.74 req/s warm, with matching outputs.

Known issue

• P2P prefetches for multiple endpoint shards currently start serially within one scheduler poll. This affects latency rather than KV correctness and is tracked in #424.

Full Changelog: https://github.com/novitalabs/pegaflow/compare/v0.23.6...v0.23.7