Designing AI agents that know when to step back

Designing user experience for agentic AI: A framework for human-AI coordination - Amazon Science

Close

Close

Social

bluesky

threads

twitter

instagram

youtube

facebook

linkedin

github

rss

Menu

Research

Research areas

Automated reasoning

Cloud and systems

Computer vision

Conversational AI

Economics

Information and knowledge management

Machine learning

Operations research and optimization

Quantum technologies

Robotics

Search and information retrieval

Security, privacy, and abuse prevention

Sustainability

Our scientific contributions

Publications

Research from our scientists and collaborators.

Conferences

Our experts present and discuss cutting-edge research at scientific meetings globally.

Research areas

Automated reasoning

Cloud and systems

Computer vision

Conversational AI

Economics

Information and knowledge management

Machine learning

Operations research and optimization

Quantum technologies

Robotics

Search and information retrieval

Security, privacy, and abuse prevention

Sustainability

Our scientific contributions

Publications

Research from our scientists and collaborators.

Conferences

Our experts present and discuss cutting-edge research at scientific meetings globally.

News & blog

The latest from Amazon researchers

Amazon Science Blog

Technical deep-dives and perspectives from our scientists.

News

Research milestones and recent achievements.

The latest from Amazon researchers

Amazon Science Blog

Technical deep-dives and perspectives from our scientists.

News

Research milestones and recent achievements.

Collaborations

Amazon Research Awards

Overview

Call for proposals

Latest news

Research stories

Recipients

Amazon Nova AI Challenge

Overview

Rules

FAQs

Teams

Research collaborations

Overview

Carnegie Mellon University

Columbia University

Hampton University

Howard University

IIT Bombay

Johns Hopkins University

Max Planck Society

MIT

Tennessee State University

University of California, Los Angeles

University of Illinois Urbana-Champaign

University of Southern California

University of Texas at Austin

Virginia Tech

University of Washington

Amazon Research Awards

Overview

Call for proposals

Latest news

Research stories

Recipients

Amazon Nova AI Challenge

Overview

Rules

FAQs

Teams

Research collaborations

Overview

Carnegie Mellon University

Columbia University

Hampton University

Howard University

IIT Bombay

Johns Hopkins University

Max Planck Society

MIT

Tennessee State University

University of California, Los Angeles

University of Illinois Urbana-Champaign

University of Southern California

University of Texas at Austin

Virginia Tech

University of Washington

Resources

Code and datasets

AGI Labs

Meet the team building useful AI agents.

Amazon Nova

Try Amazon’s frontier foundation models.

Code and datasets

AGI Labs

Meet the team building useful AI agents.

Amazon Nova

Try Amazon’s frontier foundation models.

Careers

Careers

Explore our open roles.

Amazon Scholars

Faculty research opportunities on industry-scale technical challenges.

Postdoctoral Science Program

Early-career research opportunities alongside experienced industry scientists.

Careers

Explore our open roles.

Amazon Scholars

Faculty research opportunities on industry-scale technical challenges.

Postdoctoral Science Program

Early-career research opportunities alongside experienced industry scientists.

Search

Submit Search

Conversational AI

Designing AI agents that know when to step back

As AI agents become more autonomous, the key challenge isn't what they can do; it's how to design the human side of the equation.

By James Pierce , Siddharth Gupta , Vaiva Kalnikaitė

March 11, 2026

7 min read

Share

Share

Copy link

Email

X

LinkedIn

Facebook

Line

Reddit

QZone

Sina Weibo

WeChat

WhatsApp

分享到微信

x

Agentic AI is taking off, and for good reason. AI agents can now write code, conduct research, plan travel, handle customer service, and more. Yet amid the excitement about what AI agents can do, a key question has been neglected: how do we design the human side of the equation? That question is critical, because agentic AI isn’t just another feature to bolt onto existing products. It’s a fundamentally different kind of software that demands fresh thinking. Unlike traditional software, agentic AI can be proactive and conversational, sometimes even anthropomorphic. It doesn’t just respond to commands; it initiates actions and makes decisions autonomously.

Agentic AI isn't just another feature to bolt onto existing products. It's a fundamentally different kind of software that demands fresh thinking.

This capability is what makes agentic AI so useful, but it’s also what makes effective interactions hard to design. A central user-experience (UX) challenge is coordination: the interplay between what users do, what they experience, and what the AI is doing, both visibly and behind the scenes. Trust, control, and transparency are essential to the agentic-AI user experience, and they all depend on getting this coordination right. Here, we introduce a framework for thinking about human-AI coordination. We also offer a vocabulary for characterizing agentic experiences, including when the AI feels too absent, too intrusive, or appropriately calibrated.

A framework for human-AI coordination

One of the most critical decisions in AI UX design is how visible and interactive AI capabilities should be. Should users direct the agent step by step, let it act autonomously, or work somewhere in-between? And how should this change based on the task, the user’s expertise, and the current context? You can think of coordination along these three dimensions:

Human involvement: how much effort and attention the user invests in directing or monitoring the AI; AI salience: how prominent the AI feels in the experience (for example, a conversational chatbot with a name and persona has high salience, autocomplete suggestions have lower salience, and AI-generated navigation menus and backend optimizations have little or none); AI activity: what the AI is doing, whether or not the user sees it.

Coordination is about aligning these dimensions. When human involvement and AI salience are both low, coordination is light-touch. When they are high, coordination is more hands-on. The right balance is often somewhere in-between, with an awareness of what the AI is doing in the background.

Three zones of…