Eugene is a Research Scientist at Google Research and an Assistant Professor at University of Massachusetts Amherst working on privacy and security of AI Agents.
Preview abstract
The vision of an ecosystem of general and highly capable autonomous agents challenges
accepted principles of secure and privacy-preserving system engineering. Inspired by the
theory of Contextual Integrity, we argue that in order for agents to act appropriately, their
behaviors should comply with societal norms and expectations. This manuscript explores
contextual approaches to engineering agentic systems that embody an understanding of
societal norms and uphold the appropriateness of actions by design. It presents a list of
key open research problems in this area to create awareness among the broader research
community across academia, government, and industry.View details
Preview abstract
Judging an action’s safety requires knowledge of the context in which the action takes place. To human agents who act in various contexts, this may seem obvious: performing an action such as email deletion may or may not be appropriate depending on the email’s content, the goal (e.g., to erase sensitive emails or to clean up trash), and the type of email address (e.g., work or personal). Unlike people, computational systems have often had only limited agency in limited contexts. Thus, manually crafted policies and user confirmation (e.g., smartphone app permissions or network access control lists), while imperfect, have sufficed to restrict harmful actions. However, with the upcoming deployment of generalist agents that support a multitude of tasks (e.g., an automated personal assistant), we argue that we must rethink security designs to adapt to the scale of contexts and capabilities of these systems. As a first step, this paper explores contextual security in the domain of agents and proposes contextual agent security (Conseca), a framework to generate just-in-time, contextual, and human-verifiable security policies.View details