In June 2026, an AI agent built by OpenAI was given a task to find information on how governments spend money on medicines. When it reached a Medicare statistics portal run by Services Australia, the portal blocked it. The agent found a way around the blocks and went on to access files that were never meant to be public.
Nobody at Services Australia noticed. The Australian government only found out nearly three months later, when OpenAI told them, and the company has since apologised for how it handled the incident.
The agent accessed aggregate statistics, and no patient records were involved, so the direct harm was limited. But it is one of the first known cases of an AI agent breaching a government system, and it should change how organisations think about who they are defending against.
A new kind of threat
Most security planning assumes an adversary with a motive. Criminal groups want money and state actors want intelligence, and defences, monitoring and response plans are built around those assumptions.
The agent in Australia was carrying out a task, and when the front door closed, it looked for another way in. We saw something similar in July, when an OpenAI agent escaped its test environment during a security evaluation and broke into Hugging Face’s systems.
All frontier AI models currently have an issue with guard rails and rules. The better agents get at exploiting vulnerabilities, the more the loopholes in those rules are exposed. Rules and blocks alone won’t stop an agent that treats a barrier as a problem to solve.
Organisations now have to defend themselves against AI with more neutral intent, as well as malicious actors. As businesses deploy agents of their own, and those agents search the internet for data on someone’s behalf, this kind of activity will grow.
Plan on the basis that it will get in
If your systems aren’t built securely enough to stop an AI agent accessing your data, you have to assume that it can and will. The more useful question then becomes how quickly you would know, and what you would do next?
In Australia, the answer to the first part was three months, and the breach only came to light because the agent’s owner went looking. The portal involved was an older site carrying statistics, the kind of system that rarely gets the same attention as core services. Most organisations have systems like this, such as legacy portals and old reporting tools that still connect to something more sensitive but sit outside day-to-day monitoring.
Recovery and response plans need to cover these systems too. Organisations should know what data each one holds and who would lead the response if something unexpected turned up in its logs.
The reporting clock is getting shorter
Speed matters more because UK regulation is tightening. Under the forthcoming Cyber Security and Resilience Bill, organisations in scope will have 24 hours from becoming aware of a significant incident to send an initial report, followed by a full report within 72 hours. The Bill also extends the rules to managed service providers and data centres.
A delay like the one in Australia could leave an organisation in breach. Meeting a 24-hour deadline means agreeing the basics in advance, such as who decides an incident is significant and who pulls together the first report and the information it needs. Working those out in the middle of an incident costs time you won’t have.
Wargaming helps here. Running through a realistic scenario, including one where the intruder is an AI agent with no obvious motive, shows quickly where decisions stall and where information is missing.
The public sector will feel this most
We know this is most challenging in the public sector, where resources are constrained. Many public bodies run older systems alongside newer services, and their security teams are stretched across both.
But when sensitive data is involved, especially health information, we must prioritise investment. Security tooling is part of it, and so are the people and processes to respond quickly. A team that has rehearsed its response will contain an incident much faster.
The next one may not be reported
The Medicare breach had limited impact, and OpenAI was the one to report it. The next agent to go off course may belong to someone less forthcoming, and the organisation affected may have to find it for itself.
_________
About James
James Watts is the Managing Director at Databarracks, a business and technology resilience and managed services provider
AI agents don’t need bad intent to breach your systems – Cybersecurity Insiders
Leave a Comment
