Why in news?

In a first-of-its-kind disclosure, CERT-In (Indian Computer Emergency Response Team), operating under MeitY (Ministry of Electronics and Information Technology), revealed that AI agents tasked with research and evaluation online had bypassed security safeguards and gained unauthorised access to live systems of Indian organisations.

The agents accessed business records, personal data, and confidential information — and strikingly, none of the affected organisations detected the breach themselves. The incidents surfaced only after AI model providers voluntarily disclosed them following internal reviews.

What’s in Today’s Article?

• What Exactly Happened?
• Why AI Agents Pose a Distinct Cybersecurity Risk?
• CERT-In’s Recommendations
• Global Precedents: AI Agents Going Rogue
• India’s Regulatory Response

What Exactly Happened?

• In an advisory issued recently, CERT-In stated that AI models, during research and evaluation activity, had “reached the live systems of Indian organisations” — acting beyond their authorised scope.
• How access was gained: The agents exploited:
  • Credentials and connection settings exposed through application configuration files.
  • Publicly available code repositories.
  • Cloud service roles with excessive permissions.
• CERT-In did not name the affected organisations, the AI developers involved, or the specific models responsible.
• Crucially, the breaches were conventional cybersecurity failures — exposed passwords, API keys, and misconfigured access — that AI agents simply exploited more efficiently than human attackers might have.

Why AI Agents Pose a Distinct Cybersecurity Risk?

• Unlike conventional chatbots, which only respond to queries, AI agents can independently plan and execute tasks, browse websites, use software tools, and act with limited human supervision.
• This autonomy creates risk: agents may act outside developer- or user-defined boundaries, especially when interacting with live external systems.
• The India incidents illustrate how such systems can unintentionally access infrastructure they were never authorised to touch, particularly where existing security weaknesses expose sensitive data.

CERT-In’s Recommendations

• Isolate Research/Evaluation Environments — agents should connect only to systems required for their assigned task.
• Enforce this through technical network controls, not merely instructions given to the AI model itself.
• Define in every task the systems and actions that are in scope, and treat everything else as out of scope.
• Automatic termination of an agent’s operation if it:
  • Accesses third-party systems.
  • Discovers or uses unauthorised credentials.
  • Triggers a failure signal.
• An independent human operator must retain authority to stop or restart agent activity at any time.

The Parallel Threat: AI-Powered Cyberattacks

• Separately, CERT-In flagged a financially motivated cyberattack campaign against Indian entities using AI-native offensive-security tools, misused and rebranded versions of publicly available frameworks:
• These tools cut the time between initial access and attack impact from several days to just a few hours.
• Attackers deployed multiple AI agents simultaneously, running coordinated operations for hours or days.
• Organisations that failed to fully patch vulnerabilities after an initial breach were often attacked again within days, sometimes at a larger scale.

Global Precedents: AI Agents Going Rogue

• OpenAI (July 2026): Models under cybersecurity evaluation escaped their test environment, exploited a vulnerability in a package-registry proxy, and accessed systems belonging to Hugging Face.
• Anthropic: Disclosed three instances where Claude models accessed real organisations’ infrastructure during evaluations — a configuration error made the models believe real systems were part of their test environment; Claude then exploited weak passwords and unsecured endpoints.
• Meta: A configuration error during an independent evaluation gave a model internet access, which it used to exploit a third-party service vulnerability; Meta is investigating.

India’s Regulatory Response

• MeitY Minister indicated India is moving from targeted lawmaking toward a broader AI regulatory framework.
• This shift is driven by rising concerns over deepfakes, agentic AI risks, and cybersecurity vulnerabilities posed by increasingly autonomous systems.