microsoft/waza

microsoft/waza

Description: CLI / Framework for Agent Skills - create, test, measure and improve skill quality and effectiveness

Language: Go

License: MIT

Stars: 990

Forks: 56

Open issues: 16

Created: 2026-02-27T21:39:05Z

Pushed: 2026-06-06T12:08:53Z

Default branch: main

Fork: no

Archived: no

README:

Waza

A Go CLI for evaluating AI agent skills — scaffold eval suites, run benchmarks, and compare results across models.

📖 [Getting Started / Docs](https://microsoft.github.io/waza/)

Installation

Binary Install (recommended)

Download and install the latest pre-built binary with the Bash install script on macOS, Linux, or Windows Bash environments such as Git Bash, MSYS2, or Cygwin:

curl -fsSL https://raw.githubusercontent.com/microsoft/waza/main/install.sh | bash

The Bash script auto-detects the OS and architecture of the environment where Bash is running (linux/darwin/windows, amd64/arm64), downloads the latest standalone waza CLI release, verifies the checksum, and installs to /usr/local/bin (or ~/bin if not writable).

For native Windows PowerShell:

irm https://raw.githubusercontent.com/microsoft/waza/main/install.ps1 | iex

The PowerShell script downloads the latest standalone native Windows waza binary, verifies the checksum, and installs to an existing waza.exe location or %LOCALAPPDATA%\Microsoft\Waza. On Windows, piping the Bash command from PowerShell may invoke WSL and install the Linux binary inside WSL.

Or browse the GitHub Releases page and choose the standalone waza binary assets for the version you want.

Install from Source

Requires Go 1.26+:

Note: due to the use of LFS artifacts you cannot install waza using go install. To install waza outside of a normal release, clone the repository:

git clone https://github.com/microsoft/waza.git
cd waza

# ensure git LFS-based artifacts are available (for embedded copilot binaries)
git lfs install
git lfs pull

go build -o waza ./cmd/waza
./waza

Waza bundles the GitHub Copilot CLI used by the copilot-sdk executor and extracts it to the local user cache on first use. Set COPILOT_CLI_PATH only when you need to force a specific Copilot CLI binary.

Azure Developer CLI (azd) Extension

Waza is also available as an azd extension:

# Add the waza extension registry
azd ext source add -n waza -t url -l https://raw.githubusercontent.com/microsoft/waza/main/registry.json

# Install the extension
azd ext install microsoft.azd.waza

# Verify it's working
azd waza --help

Once installed, all waza commands are available under azd waza. For example:

azd waza init my-eval --interactive
azd waza run examples/code-explainer/eval.yaml -v

Update Notifications

Waza automatically checks for new versions in the background. If an update is available, a notice appears after command output:

A newer version of waza is available: v0.24.0 → v0.28.0. Run: waza update

Run waza update to download and execute the official OS-specific installer after an explicit confirmation prompt. It uses the Bash installer on macOS/Linux and the PowerShell installer on native Windows. Use waza update --yes to skip the prompt in scripted environments. The check is non-blocking (never slows commands), cached for 24 hours, and can be disabled with --no-update-check or WAZA_NO_UPDATE_CHECK=1.

Quick Start

For New Users: Get Started in 5 Minutes

See [Getting Started Guide](docs/GETTING-STARTED.md) for a complete walkthrough:

# Initialize a new project
waza init my-project && cd my-project

# Create a new skill
waza new skill my-skill

# Define the skill in skills/my-skill/SKILL.md
# Write evaluation tasks in evals/my-skill/tasks/
# Add test fixtures in evals/my-skill/fixtures/

# Run evaluations
waza run my-skill

# Check skill readiness
waza check my-skill

All Commands

# Build
make build

# Initialize a project workspace
waza init [directory]

# Update waza to the latest release
waza update

# Create a new skill
waza new skill skill-name

# Create a new eval scaffold from an existing SKILL.md
waza new eval skill-name

# Generate a task YAML by recording a prompt run
waza new task from-prompt "Explain this code and suggest fixes" evals/code-explainer/tasks/recorded-task.yaml

# Check if a skill is ready for submission
waza check skills/my-skill

# Suggest an eval suite from SKILL.md
waza suggest skills/my-skill --dry-run
waza suggest skills/my-skill --apply

# Note: 'generate' is available as an alias for 'new' (see below for new command)
# Note: Custom agents (.agent.md) are supported — see https://microsoft.github.io/waza/guides/custom-agents/

# Run evaluations (works with both skills and custom agents)
waza run examples/code-explainer/eval.yaml --context-dir examples/code-explainer/fixtures -v

# Grade output from a previous `waza run --output results.json ...`
waza grade eval.yaml --results results.json

# Compare results across models
waza compare results-gpt4.json results-sonnet.json

# Generate eval coverage grid
waza coverage --format markdown

# Count tokens in skill files
waza tokens count skills/

# Compare skill token budgets vs main
waza tokens compare main --skills --threshold 10

# Suggest token optimizations
waza tokens suggest skills/

Commands

waza update

Update waza to the latest release by running the official OS-specific installer after confirmation.

| Flag | Description | |------|-------------| | --yes, -y | Skip the confirmation prompt |

Example:

waza update
waza update --yes

waza init [directory]

Initialize a waza project workspace with separated skills/ and evals/ directories. Idempotent — creates only missing files.

| Flag | Description | |------|-------------| | --no-skill | Skip the first-skill creation prompt |

Creates:

• skills/ — Skill definitions directory
• evals/ — Evaluation suites directory
• .github/workflows/eval.yml — CI/CD pipeline for running evals on PR
• .gitignore — Waza-specific exclusions
• README.md — Getting started guide for your project

Example:

waza init my-project
# Optionally creates first skill interactively

waza init my-project --no-skill
# Skip skill creation prompt

waza new skill

Create a new skill with…