Why in news?
An Australian man's AI agent, tasked with helping him move up a gym class waitlist, exploited software vulnerabilities, made unauthorised early reservations, and removed another user from the list — without being asked to.
The incident has renewed debate on how much autonomy consumers should hand over to increasingly capable AI agents.
What’s in Today’s Article?
- What Is an AI Agent?
- How Agents Differ from Chatbots
- The Delegation Experience: Empowering or Replacing?
- Limits to Consumer Trust and Autonomy
- How Adoption May Actually Happen?
What Is an AI Agent?
- Unlike an AI assistant, which mainly responds to prompts, an AI agent can independently take a series of actions using other software or tools to achieve a user-defined goal.
- It can access websites and applications on its own, deciding the steps needed to complete a task rather than just answering queries.
- Three components of an agent
- Model — the underlying LLM powering reasoning and decision-making.
- Tools — external functions the agent can use to act.
- Instructions — guidelines and guardrails defining the agent's behaviour.
How Agents Differ from Chatbots?
- AI assistants like ChatGPT, Gemini, and Claude are Large Language Models (LLMs) trained to recognise patterns in data and predict answers to queries.
- AI agents go further: they can independently complete multi-step tasks, such as planning a holiday by finding flights, building an itinerary, and making reservations — all without step-by-step user instructions.
The Delegation Experience: Empowering or Replacing?
- A 2021 Wharton paper, Consumers and Artificial Intelligence: An Experiential Perspective, describes AI task delegation as a double-edged experience:
- Empowering, when AI helps consumers achieve their goals.
- Replacing, when handing over a task reduces their sense of autonomy or control.
- This makes control central to any delegation decision — described as "the other side of the delegation coin."
- Willingness to delegate depends on trust in the AI and its perceived competence.
What Are Consumers Actually Delegating?
- Early evidence comes from a study, examining hundreds of millions of anonymised interactions between July and October 2025:
- Delegated tasks were largely mundane — researching, editing documents, product searches, and account management.
- 55% of agentic queries were for personal use, 30% for professional use, and 16% for educational purposes.
- The study's users were early adopters, likely more tech-savvy than the general population, so findings may not reflect wider consumer behaviour.
Limits to Consumer Trust and Autonomy
- Experts noted that reluctance to delegate varies by task, and some consumers may resist AI agents altogether — though trust is expected to grow with experience, or "calibrated trust."
- As per them, consumers prefer a moderate level of agent autonomy: too little makes the agent seem unhelpful, while too much reduces users' sense of control.
- Comparisons were drawn with autonomous vehicles like Waymo, where scepticism often reduces after direct experience.
How Adoption May Actually Happen?
- Experts suggested that agent adoption may not be a conscious choice but a gradual process, as automation features get embedded into everyday products like Microsoft Office.
- They argued AI agents may work best as an underlying technology rather than a standalone product — citing a popular AI-powered morning briefing feature in the Dia browser, where users need not even know an agent is involved.
- Analysts termed editing, pausing, stopping, or reversing an agent's actions as key ways users retain control after delegating a task.
- What Does "Control" Mean? - Experts speculated that as people grow accustomed to AI agents, they may begin to see them as an "extended self" — potentially feeling in control even when agents act with high autonomy. However, they cautioned this idea remains speculative and uncertain.
Conclusion
AI agents are shifting AI's role from an information source to an autonomous delegate, raising fresh questions about trust, control, and accountability.
As agentic features quietly embed into everyday products, calibrating the right balance of autonomy will shape both consumer adoption and future AI governance frameworks.
