research area Human-Computer Interaction
content type paperpublished January 2026

AgentBuilder: Exploring Scaffolds for Prototyping User Experiences of Interface Agents

AuthorsJenny T. Liang†, Titus Barik, Jeffrey Nichols, Eldon Schoop, Ruijia Cheng

View publication

Interface agents powered by generative AI models (referred to as “agents”) can automate actions based on user commands. An important aspect of developing agents is their user experience (i.e., agent experience). There is a growing need to provide scaffolds for a broader set of individuals beyond AI engineers to prototype agent experiences, since they can contribute valuable perspectives to designing agent experiences. In this work, we explore the affordances agent prototyping systems should offer by conducting a requirements elicitation study with 12 participants with varying experience with agents. We identify key activities in agent experience prototyping and the desired capabilities of agent prototyping systems. We instantiate those capabilities in the AgentBuilder design probe for agent prototyping. We conduct an in situ agent prototyping study with 14 participants using AgentBuilder to validate the design requirements and elicit insights on how developers prototype agents and what their needs are in this process.

  • † Carnegie Mellon University

Related readings and updates.

CAMPHOR: Collaborative Agents for Multi-Input Planning and High-Order Reasoning On Device

October 15, 2024research area Methods and Algorithms, research area Speech and Natural Language Processing

While server-side Large Language Models (LLMs) demonstrate proficiency in tool integration and complex reasoning, deploying Small Language Models (SLMs) directly on devices brings opportunities to improve latency and privacy but also introduces unique challenges for accuracy and memory. We introduce CAMPHOR, an innovative on-device SLM multi-agent framework designed to handle multiple user inputs and reason over personal context locally, ensuring…

Read more

Towards Learning Multi-Agent Negotiations via Self-Play

January 28, 2019research area Computer VisionWorkshop at ICCV

Making sophisticated, robust, and safe sequential decisions is at the heart of intelligent systems. This is especially critical for planning in complex multi-agent environments, where agents need to anticipate other agents’ intentions and possible future actions. Traditional methods formulate the problem as a Markov Decision Process, but the solutions often rely on various assumptions and become brittle when presented with corner cases. In…

Read more