microsoft/LAB521-Improving-Agent-Behavior-Using-Reinforcement-Learning-from-Traces-NEW

microsoft/LAB521-Improving-Agent-Behavior-Using-Reinforcement-Learning-from-Traces-NEW

License: MIT

Stars: 1

Forks: 1

Open issues: 0

Created: 2026-06-23T20:09:22Z

Pushed: 2026-06-25T17:23:52Z

Default branch: main

Fork: no

Archived: no

README: --- lab: title: TechLab: Improving Agent Behavior Using Reinforcement Learning from Traces description: level: 400 duration: 120 minutes islab: true primarytopics: ---

@lab.Title

Introduction

> [!NOTE] > This is a 75-minute hands-on lab where you will improve AI agent behavior using reinforcement fine-tuning (RFT) on traces from an agentic workflow.

Learning Objectives

By the end of this lab, you should be able to:

• Understand how reinforcement fine-tuning (RFT) improves agentic tool-calling behavior without redesigning the agent
• Inspect agent execution traces and define "good behavior" using evaluation graders with partial credit scoring
• Build training data from agent runs and submit an RFT fine-tuning job to Microsoft Foundry
• Evaluate checkpoints to identify the best-performing fine-tuned model

Resources

The repository to the lab is live, you can find it at [GitHub Repo](https://aka.ms/build2026-LAB521)

> [!TIP] > You can find your Microsoft Foundry logins, and any other credentials in the Resources tab.

Lab Outline

The lab is organized into 5 notebooks, taking you through the full RFT workflow:

1. Introduction & Setup - Connect to Microsoft Foundry and verify your environment 2. Meet the Agent - Run Zava's return-resolution agent live on scenarios and observe its tool-calling behavior 3. Baseline & Grader - Evaluate the base model, understand how the grader drives RFT learning, and calibrate the pass threshold 4. Build Data & Submit a Job - Explore the RFT data format, craft a training example, and submit a fine-tuning job 5. Training Results & Evaluate - Analyze the reward curve, compare checkpoints, and run a head-to-head evaluation of the fine-tuned model

Business Scenario

In this lab you'll be improving an AI agent for Zava, a fictional DIY retailer that operates both online and in physical stores across the United States. Zava specializes in home improvement, hardware, tools, and DIY supplies.

The Challenge

Zava's customer service team handles a high volume of return, exchange, and dispute requests. Each request requires the agent to:

1. Look up the customer's order via a tool call 2. Apply a complex return policy with multiple rules (loyalty tiers, product categories, sale flags, late delivery credits, restocking fees, etc.) 3. Determine the correct resolution - refund, store credit, exchange, or denial - and compute the exact dollar amounts

The base model o4-mini handles most cases correctly, but struggles with edge cases: it scores around 73% on the validation set. The goal of this lab is to push that to 87%+ using RFT.

Why Reinforcement Fine-Tuning?

Unlike supervised fine-tuning (SFT), RFT does not require labeled ideal responses. Instead:

• You provide training prompts and a grader function that scores model outputs
• During training, the model generates candidate responses and receives reward signals
• The model learns through trial and error to maximize its score on the grader

This makes RFT particularly well-suited to agentic tasks where "correctness" is defined by a business rule - not by a single golden response.

The Grader

The grader in this lab evaluates each model response on four dimensions:

| Dimension | Weight | What it checks | |-----------|--------|----------------| | Correct action | 40% | Did the model choose refund / denial / store credit / exchange correctly? | | Correct amounts | 30% | Are the dollar figures (refund, fees, credits) computed correctly? | | Policy reasoning | 20% | Does the model cite the correct policy rules? | | Tool usage | 10% | Did the model call *get_order* to retrieve order data? |

The grader returns a score between 0.0 and 1.0. A pass threshold of 0.80 is used: responses scoring ≥ 0.80 count as "passing" for the RL reward signal.

Click Next to set up your lab environment and get started.

===

Part 1 - Get started

Sign in to Windows

As a first step, login into the lab Virtual Machine using the credentials you can find in the Resources tab under the Skillable VM name.

• Password: +++@lab.VirtualMachine(Win11-Pro-Base).Password+++

> [!TIP] > First time using Skillable? The green "T" (e.g., +++Admin+++) indicates values that are automatically input for you at the current cursor location in VM, with one click. This reduces your effort and minimizes input errors. > Also, you can always click on the images to enlarge them, if needed.

Open workshop in Visual Studio Code

In this workshop, we will be using Visual Studio Code as our development environment. It is equipped with all the necessary tools and dependencies pre-installed. This will allow you to focus on learning and prototyping without worrying about local setup.

1. To launch Visual Studio code, you can either click on the icon in the home page, or search for Visual Studio Code in the start up menu, or Click on the Visual Studio icon that is pinned to the task bar.

2. In the Visual Studio Code, open a new terminal (Terminal → New Terminal) and run the setup script to populate your .env file with your Microsoft Foundry credentials:

./src/scripts/get-env

> [!TIP] > Ensure the default terminal launched is bash. If not, switch the terminal by clicking the down arrow, next to the add icon and select bash to ensure the default terminal is bash.

This will create a .env file in the repo root with values for:

AZURE_OPENAI_ENDPOINT=https://.openai.azure.com/openai/v1 AZURE_OPENAI_API_KEY= FOUNDRY_PROJECT_ENDPOINT=https://.services.ai.azure.com/api/projects/

Login to Azure

1. Next, open the edge browser and navigate to Python 3.13....**, you will have successfully executed the notebook.

What have you achieved?

As you work through the cells in 01-introduction-setup.ipynb in order, you will see:

1. Install dependencies - Installs required Python packages (openai, python-dotenv, requests, matplotlib, tabulate). This may take 1-2 minutes. 2. Import libraries and connect to Microsoft Foundry - Loads your .env file and creates an OpenAI client pointed at your Microsoft Foundry endpoint. 3. Verify your environment -...