amazon/gpt-oss-20b-p-eagle-long-context

Model Overview

P-EAGLE is a parallel-drafting speculative decoding model that generates K draft tokens in a single forward pass. It transforms EAGLE—the state-of-the-art speculative decoding method—from autoregressive to parallel draft generation.

For use cases of less than 10k context length - please consider using gpt-oss-20b-P-Eagle.

Model Details

The model architecture is illustrated in the following figure. Specifically, we trained a 4-layer P-EAGLE for GPT-OSS 20B as the target model, with number of parallel-token prediction as 10.

P-EAGLE follows the vanila EAGLE 3 using three layers of hidden states from the target model.

Model Description

• Developed by: AWS
• Model type: EAGLE
• Language(s) (NLP): English
• License: Apache License 2.0
• Target model: GPT-OSS 20B

Model Sources

• Paper: P-EAGLE: Parallel-Drafting EAGLE with Scalable Training

Training Data

• [Ultrachat_200k](HuggingFaceH4/ultrachat_200k)

Similar to nvidia/gpt-oss-120b-Eagle3-long-context: only prompts from the datasets were used for data synthesis (the original responses from GPT were not used for data synthesis) which is then used to train the P-Eagle.

Usage

To serve the checkpoint in vLLM:

> Note: GPT-OSS 20B uses hybrid attention (sliding window + full attention). When combined with the P-EAGLE drafter, a KV cache grouping fix is required for vLLM to correctly separate speculator layers into a dedicated KV cache group. Without this fix, vLLM will fail with a validate_same_kv_cache_group error. Apply the fix from the PR or use a vLLM version that includes it.

vllm serve openai/gpt-oss-20b \
--speculative-config '{"method": "eagle3", "model": "amazon/gpt-oss-20b-p-eagle-long-context", "num_speculative_tokens": 7, "parallel_drafting": true}' \
--tensor-parallel-size 1 \
--kv-cache-dtype fp8 \
--async-scheduling \
--stream-interval 20 \
--max-cudagraph-capture-size 4096 \
--no-enable-prefix-caching \
--port 8050 \
--gpu-memory-utilization 0.9 \
--max-num-seqs 128 \
--max-model-len 131072

Evaluation

From vllm-bench, with speculation length of 7 and max-new-token of 2048, we see the following acceptance length for [[gpt-oss-20b-P-Eagle]](https://huggingface.co/amazon/GPT-OSS-20B-P-EAGLE) and its long-context extension (the current model).

| Benchmark | gpt-oss-20b-P-Eagle | long context extension | |-------------|------:|-------------:| | aa-lcr (100k) | 1.30 | 2.25 | | humaneval | 2.79 | 2.78 | | mtbench | 2.39 | 2.31 |

The command to run benchmarking is shown as below.

vllm bench serve \
--backend openai-chat \
--base-url http://localhost:8050 \
--endpoint /v1/chat/completions \
--model openai/gpt-oss-20b \
--dataset-name custom \
--dataset-path /home/ubuntu/eval_datasets/humaneval_custom.jsonl \
--custom-output-len 2048 \
--num-prompts 164 \
--max-concurrency 1 \
--request-rate inf \
--temperature 0 \
--save-result \
--save-detailed \

Ciatation

@article{hui2026p,
title={P-EAGLE: Parallel-Drafting EAGLE with Scalable Training},
author={Hui, Mude and Huang, Xin and Salas, Jaime Campos and Sun, Yue and Pemberton, Nathan and Song, Xiang and Khetan, Ashish and Karypis, George},
journal={arXiv preprint arXiv:2602.01469},
year={2026}
}