microsoft/Build26-BRK252-from-observability-to-roi-for-ai-agents-on-any-framework

microsoft/Build26-BRK252-from-observability-to-roi-for-ai-agents-on-any-framework

Description: Resources for modern agent observability — cross-framework tracing, evals, always-on signals connecting agent behavior to business outcomes, cost, and ROI. From Microsoft Build 2026.

License: MIT

Stars: 1

Forks: 5

Open issues: 0

Created: 2026-04-10T22:46:06Z

Pushed: 2026-06-06T05:12:15Z

Default branch: main

Fork: no

Archived: no

README:

Microsoft Build 2026

BRK252: From observability to ROI for AI agents on any framework

Nondeterministic, multi-agent systems break traditional monitoring. As agents reach production, observability must be built in—not added after failures. This session covers modern agent observability: cross-framework tracing and evals, rigorous inner-loop practices, evolving context-specific evals, and always-on signals that connect behavior to business outcomes to measure value, cost, and ROI.

| | |:---:| | [  ](https://build.microsoft.com/en-US/sessions/brk252)|

| | | |:---------|:------------| | Watch the Session Replay | BRK252: From observability to ROI for AI agents on any framework| | Complete the Hands-On Lab | LAB540: Observe, optimize and protect your hosted agents in Microsoft Foundry | | Read The Dev Blog Post | Build 2026: From observability to ROI for AI agents on any framework | | Download The Session Slides | BRK252 "Download Slides" Link From Session Page | | | |

Session Overview

Reliable AI Agent development needs observability. Agents are inherently non-deterministic, creating new challenges in reliability and consistency for developers and operators. Delivering trustworthy solutions requires a unified end-to-end observability platform that can help you understand and optimize the cost, health and behavior of agents in real time.


In this session, you will walk through the end-to-end agent devops lifecycle with a focus on using observability tools and workflows to go from prototype to production. You will get started with out-of-the-box observabilty (plan-code-test-release) - then fast forward to production (monitor, analyze, optimize) and learn to hill climb with confidence using the new Foundry Optimizer capability -- with live demos at each step that show you how to:

• observe any agent, on any framework - with tracing and evals
• get code-first observability - with skills-based guided experiences
• optimize for scale - with Foundry Optimizer to do your hill climbing
• prove the value of your product - with Agent ROI using portal and API


Learning Outcomes

By the end of this session you will be able to:

• Apply cross-framework tracing and evaluation techniques to gain visibility into nondeterministic, multi-agent systems
• Establish rigorous inner-loop practices that build observability into agents before they reach production
• Design and evolve context-specific evals that keep pace with changing agent behavior and requirements
• Connect always-on production signals to business outcomes to measure agent value, cost, and ROI
• Optimize for scale with a unified end-to-end observability capabilities

Keep Learning with Copilot

> _Try these prompts in GitHub Copilot Chat to explore topics in more detail. Open a new chat session in Visual Studio Code (Ctrl+Alt+I on Windows/Linux, Cmd+Shift+I on Mac) and use a prompt below as a starting point - or write your own! 💡 Tip: connect the [Microsoft Learn MCP Server](#-microsoft-learn-mcp-server) first to ground responses in official docs.

| Topic | Prompt | |:---|:---| | Distributed tracing in Microsoft Foundry | Walk me through how to enable distributed tracing for an AI agent in Microsoft Foundry, including how to view thread results and spans across tools and model calls. Reference Trace and observe AI agents in Microsoft Foundry and Observability in the Agent Framework | | Cross-framework observability| Explain the core observability capabilities in Microsoft Foundry and how they cover the three stages of the AI application lifecycle (ideate, build, operate) for multi-agent systems. Ground your answer in Observability in generative AI.| | Evals from inner loop to production| Show me how to run cloud evaluations with the Microsoft Foundry SDK and connect trace data to evaluation results so I can evolve context-specific evals as my agent changes. Use Run evaluations in the cloud by using the Microsoft Foundry SDK as the source. | | *Connecting observability to ROI**| How do I use the Agent Monitoring Dashboard and fleet-wide monitoring in Microsoft Foundry together with cost optimization to measure agent value, cost, and ROI in production? Reference Monitor agents with the Agent Monitoring Dashboard, Monitor agent health and performance across your fleet, and Optimize model cost and performance. | | | |

Technologies Used

1. Microsoft Foundry — the unified platform for building, deploying, and managing AI agents 1. Microsoft Foundry Control Plane — manage and monitor agent fleets at scale across environments 1. Microsoft Foundry Observability — tracing, evaluations, and…