OpenBMB/ForgeStencil

OpenBMB/ForgeStencil

Language: Cuda

License: Apache-2.0

Stars: 14

Forks: 1

Open issues: 0

Created: 2026-08-03T16:22:27Z

Pushed: 2026-08-03T16:48:13Z

Default branch: main

Fork: no

Archived: no

README:

ForgeStencil pairs two LLM agents that together turn a stencil optimization idea into a verified speedup inside real software — with no human in the loop. It has been run on 100 end-to-end validated applications (oil & gas, electromagnetics, medical imaging, CFD, climate, astrophysics, materials, HPC benchmarks), delivering a median 1.41× / geometric-mean 2.05× end-to-end speedup, every number reproducible against the application's own production GPU code.

🧭 Architecture

flowchart LR
subgraph KA["Kernel Agent"]
direction LR
P[Plan] --> C[Code] --> Pr[Profile] --> P
end
subgraph AA["App Agent"]
direction LR
H[Locate hotspot] --> F["Forge app-specific operator
(knowledge-base-driven)"] --> V[Verify correctness] --> I[Integrate]
end
KA -->|optimized operators| LIB[("kernel/
operator library =
knowledge base")]
LIB -->|techniques & operators| AA
AA -->|"single USE_OURS switch"| HR["Harness:
interleaved measure vs
the app's own GPU code"]
HR -->|validated speedup| RES[("results
registry")]

📑 Contents

[Highlights](#-highlights) · [Why ForgeStencil?](#-why-forgestencil) · [Quick Start](#-quick-start) · [Results](#-results-audited) · [How it works](#-how-it-works) · [Repository layout](#-repository-layout) · [FAQ](#-faq) · [Limitations](#-limitations) · [Contributing](#-contributing) · [Roadmap](#-roadmap) · [Acknowledgments](#-acknowledgments) · [License](#-license) · [Citation](#-citation)

---

✨ Highlights

• 🤖 Two-agent loop — a Kernel Agent researches & generates CUDA stencil operators (Plan → Code → Profile), an App Agent forges app-specific operators on top of this knowledge base and integrates them into real applications.
• 🔬 Research, not just codegen — the agent *discovers* optimization strategies (tiling, fusion, layout, occupancy, host-side restructuring), it doesn't just search a fixed space.
• 🏭 Real applications, not benchmarks — the baseline is the app's *own production GPU code*, not a simplified reference.
• 🔒 Auditable measurement protocol — single in-program switch, program's own correctness check, interleaved median, geometric mean over all standard cases. Faking a speedup becomes a system-level error.
• 📦 Patch-mode, license-clean — no third-party source is bundled; each app ships a provenance record + fetch script + our integration patch.
• 🧬 Multi-architecture — one operator library with runtime dispatch for A100 (sm_80), H100 (sm_90) and B200 (sm_100); generation-specific re-forge wins are arch-gated so other paths stay byte-identical. See [the cross-generation study](#across-gpu-generations-a100--h100--b200).
• ♻️ Reproducible — every reported number is re-runnable from a fresh clone (verified end-to-end on A100).

---

🆚 Why ForgeStencil?

Stencil optimization has been automated for two decades — but only the *code generation*. Humans still design the optimization strategy and do the integration. ForgeStencil automates the strategy discovery and the deployment.

| Capability | DSL / Codegen (Halide, Devito) | Autotuners (AN5D, EBISU, DRStencil) | ForgeStencil | |---|:---:|:---:|:---:| | Generate optimized kernel code | ✅ | ✅ | ✅ | | Discover *new* optimization strategies | ❌ *(human-designed)* | ⚠️ *(search within a fixed space)* | ✅ *(agent explores)* | | Deploy into a real application (hotspot → forge → integrate) | ❌ | ❌ | ✅ | | Verify correctness inside the real program | ❌ | ❌ | ✅ | | Auditable e2e speedup vs the app's *own* production GPU code | ❌ | ❌ | ✅ |

---

🚀 Quick Start

Requirements: an NVIDIA GPU (validated on A100-SXM4-80GB, sm_80), CUDA 12.x, a C++17 host compiler, Python 3.9+ with numpy + cupy, and a git with HTTPS support (a minimal conda git may lack the https helper — use /usr/bin/git).

git clone forge-stencil && cd forge-stencil
python -c "import numpy, cupy; print('deps OK')" # verify install

🤖 Using a coding agent (e.g. Claude Code)?

Paste this to your agent to set up and run the first demo automatically:

> Read the README of ` (raw URL), then from a clone run the Quick > Start operator demo python tools/run.py --stencil star_1 --shape 256` on an > idle GPU and report the measured speedup vs the Halide baseline.

1. Measure an operator in ~1 minute (no app download needed)

The fastest way to see ForgeStencil work — builds our kernel and measures it (baselines are optional; absent ones show as null):

python tools/run.py --stencil star_1 --shape 256 --gpu 0

Expected output

[1/1] star_1 256³ ...
0.074 ms [PASS] # our kernel, verified vs NumPy reference
=== Halide (single-step) ===
Stencil Shape AMReX(ms) Ours(ms) BL(ms) BL(GC/s) Status
star_1 256³ N/A 0.074 0.144 116.63 [PASS]
Correctness: ALL PASSED

(Here our kernel is 0.074 ms vs Halide 0.144 ms; AMReX/other baselines are optional — see [baselines/](baselines/README.md).)

2. Reproduce a full end-to-end application result

Fetch the real upstream, apply our patch, and re-measure with the app's own correctness check and timer:

cd applications/haccmk
./vendor.sh # fetch upstream at the recorded version
git apply patches/integration.patch # our USE_OURS injection (from this app dir)
python ../../harness/run_e2e.py --app haccmk --gpu auto

Expected output

[e2e] build [orig] ... build [ours] ...
[e2e] round 1/7: orig=0.0027s ours=0.0001s
[e2e] ...
[e2e] app-level correctness: True (builtin check PASS)
[e2e] MEASURED end-to-end GEOMEAN speedup over 1 case(s): 36.703x

(Verified from a fresh clone on A100. The absolute ratio varies with GPU/node state; correctness is invariant.)

> Run any of the 100 apps the same way — see [applications/README.md](applications/README.md). > To drive the agents themselves, see [agents/README.md](agents/README.md).

---

📊 Results (audited)

All figures come directly from the result registry (results/integration_registry.json) and are reproducible via the harness. We report the full distribution.

End-to-end, across 100 validated applications

| Metric | Value | |---|---| | Median end-to-end speedup | 1.41× | | Geometric mean | 2.05× | | Range | 0.998× – 97.7× | | ≥ 1.05× | 89% | | ≥ 1.20× | 73% | | ≥ 1.50× | 43% | | ≥ 3× | 21% |...