When Interfaces Speak: Data-Aware Generative UI Harness for Active Interaction
We propose GenUI-Harness, a multi-agent harness pairing a Tool Agent for information retrieval and task execution with a GUI Coder Agent that identifies ambiguities and generates front-end code for structured interfaces.
ProofPaper ↗
Key points
- Most human-agent interaction today remains text-based.
- Natural language can impose cognitive overload, ambiguity, information chaos, and slow input for complex tasks; ephemeral generative UIs can present structured information and guide users toward task completion.
- Training the coder with reinforcement learning is challenging: verifiable rewards for interactive UI generation require costly execution, while LLM-as-a-Judge rewards are prone to reward hacking.
- We introduce UI-TAU Bench, a benchmark for active human-agent interaction through generated UI code, built on 10 real-world domain databases constructed from public data sources and based on Tau-Bench tool-use settings, with Lite (300 tasks) and Full (1,000 tasks) splits.
Sources (1)
- [1]When Interfaces Speak: Data-Aware Generative UI Harness for Active InteractionarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 02:49 AM
We propose GenUI-Harness, a multi-agent harness pairing a Tool Agent for information retrieval and task execution with a GUI Coder Agent that identifies ambiguities and generates front-end code for structured interfaces.
Most human-agent interaction today remains text-based.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 8, 2026ConwayResearch/Underdog-Saluki-27B-1.0
- Oct 8, 2026Introducing the Anthropic Cyber Mission
- Oct 7, 2026[AINews] Claude Haiku 5.5 — better than GPT-6 Luna at the same pricing
- Oct 7, 2026MIMESIS: Learning User Simulators as Training Environments for Interactive Agents
- Oct 6, 2026llm-openai-decisions 0.1a0
- Oct 6, 2026Tool-calling retrieval versus vector RAG for a small Greek--English knowledge base: accuracy and robustness to how users type Greek