microsoft/Build26-LAB540-observe-optimize-and-protect-your-hosted-agents-in-microsoft-foundry

microsoft/Build26-LAB540-observe-optimize-and-protect-your-hosted-agents-in-microsoft-foundry

Description: Workshop : Observe, optimize and protect your hosted agents in Microsoft Foundry

Language: Bicep

License: MIT

Stars: 25

Forks: 33

Open issues: 2

Created: 2026-04-02T16:14:06Z

Pushed: 2026-06-30T22:58:03Z

Default branch: main

Fork: no

Archived: yes

README:

Microsoft Build 2026

LAB540: Observe. Optimize and Protect Your Hosted Agents in Microsoft Foundry

Session Description

Modern agents can fail in ways that traditional monitoring can't catch. In this hands-on lab, learn more about how Microsoft Foundry Observability helps you move from prototype to production - with context-specific evaluation suites (auto-generated evaluators + test datasets) wired into developer workflows via skills and MCP tooling for hosted agents. Scale quality with continuous evaluations, trace-linked analysis and adaptive red teaming - and walk away with a sandbox you can use to explore additional features at your own pace.

Business Scenario

Zava Travel is a fictitious premium travel agency specializing in international travel experiences. The team has built the Zava Travel Concierge — an AI-powered multi-agent system that orchestrates specialist agents for flights, hotels, and car rentals to plan complete itineraries. The Concierge does *not* answer questions from its own knowledge — it delegates to these specialist sub-agents, each of which owns one CSV data source and exposes typed Python tools to query it. This is a new agent (with no pre-existing test datasets) and the team needs to ensure that agent responses are reliable, safe and high-quality even as demand scales and requirements change.

flowchart LR
User([Traveler])
Concierge[Zava Concierge
orchestrator]
Flights[Flight Specialist
flights.csv]
Hotels[Hotel Specialist
hotels.csv]
Cars[Car Rental Specialist
car_rentals.csv]

User -- "Plan a trip to Rome" --> Concierge
Concierge -- "search flights" --> Flights
Concierge -- "search hotels" --> Hotels
Concierge -- "search cars" --> Cars
Flights -- results --> Concierge
Hotels -- results --> Concierge
Cars -- results --> Concierge
Concierge -- itinerary --> User

Learning Objectives

In this hands-on lab, we'll see how the Microsoft Foundry Observability platform works with GitHub Copilot and Foundry skills, to simplify the developer experience and accelerate their progress from plan to prototype.

By completing this lab you will learn to:

1. Deploy a hosted agent with the Azure Developer CLI 1. Observe agent behavior with built-in traces, evaluations, and monitoring 1. Auto-generate test datasets and evaluators using the observe skill 1. Run the evaluate-optimize loop to iteratively improve cost, latency, and quality 1. Protect agents at scale with continuous evaluation and adaptive red teaming

> 📝 A note on outcomes. Everyone runs the same workflow and skills, but > your scores, recommendations, and optimized prompts will vary run to run — > responses are non-deterministic and samples are small (n ≈ 10). Focus on > learning the loop, not on matching specific numbers.

🎙️ Session Delivery

This repository will be updated continuously to reflect the latest changes to the platform and features. Want to revisit the specific version presented at a particular event? Check out the branch links below for the pinned version.

| Event | Date | Branch | |:---|:---|:---| | Microsoft Build 2026 | Jun 2-3, 2026 | msbuild26-release| | AI Engineer World Fair 2026 | Jun 29, 2026 | aiewf26-refresh | | | |

Check back soon for a detailed _session-delivery-resources_ folder with more detailed guidance for both instructors and self-guided laarners. > 🆕 Instructor Deck: [view or download a PDF version](https://speakerdeck.com/nitya/observe-optimize-and-protect-your-hosted-agents-in-microsoft-foundry?slide=6) for reference. Check back for updated resources to recreate and customize this for your own sessions.

🏫 Getting started

To complete this lab you must have:

1. An Azure subscription 1. A GitHub account (with a GitHub Copilot subscription) 1. Familiarity with Python, VS Code & Agentic AI lifecycles

_An Azure subscription and GitHub Copilot subscription are provided to in-venue attendees. Self-guided learners should bring their own._

There are three paths through this lab — pick the one that matches how you're taking it. The first two are manual, step-by-step walkthroughs you follow on your own; the third is a guided, Copilot-driven experience.

| Path | For | Start Here | |------|-----|------------| | 🏫 Skillable (in-venue)| Build attendees — Azure RG and Foundry agent are pre-provisioned with a Skillable Azure subscription | [workshop/skillable](./workshop/skillable/README.md) | | 🏠 Self-Guided (at home) | You'll provision everything yourself with azd up using your own Azure subscription. | [workshop/self-guided](./workshop/self-guided/README.md) | | 🧪 AI-Assisted (experimental) | A guided, Copilot-driven experience with inline troubleshooting that should work with either | [workshop/docs](./workshop/docs/README.md) |

---

💬 Keep Learning with Copilot

Try these prompts with GitHub Copilot to explore the topics from this session. Open Copilot Chat in VS Code (Ctrl+Alt+I on Windows/Linux, Cmd+Shift+I on Mac), paste a prompt, and see what you learn. Try connecting the [Microsoft Learn MCP Server](#-microsoft-learn-mcp-server) for the latest official documentation.

Use these as a starting point — or write your own!

| Prompt | What you'll learn | |:-------|:------------------| | *"What are Microsoft Foundry Hosted Agents, and when should I use them instead of prompt-based agents?"* | Hosted agents as containerized, framework-agnostic agent code on managed infrastructure — with per-session sandboxes, dedicated Entra identity, and a choice of Responses or Invocations protocols. | | *"What is Microsoft Foundry Observability, and how do evaluation, monitoring, and tracing...