Unified Agent: Managing Interactions across Devices

The paper argues that existing agent systems are ill-equipped for cross-device, cross-time interactions. Observations are scattered across devices and moments, yet mainstream designs either treat devices as mere tools for a single agent (lacking effective cross-time state management) or coordinate multiple agents (without maintaining a compact carried state for the standing request). The authors propose that an agent should maintain an explicitly designed state that organizes engagement evidence, stated facts, and the standing request into an action-ready form, used alongside the current observation to decide the next action.

To evaluate state designs, they construct a benchmark of user-agent interaction across devices and time. They instantiate their principle in Unified Agent and compare it against adaptations of four published agent designs. In the default setting, Unified Agent significantly outperforms these baselines. Moreover, when varying the multimodal large language model (MLLM) family, capability, and reasoning effort, Unified Agent remains ahead of all compared systems, showing the state-design advantage is robust across MLLM settings. The code and data are to be released on GitHub.