Published October 9 2026
In this blog, Ray Reyes explores what the Medicare incident reveals about AI security risk and why the real issue is not AI itself, but the identity, access and legacy-system weaknesses already sitting underneath it.
An OpenAI agent carrying out what should have been a routine research task found its way into a legacy government system, accessed information it was never intended to reach and triggered a national response. The incident has prompted a broad review of government systems and renewed debate about the risks of AI.
But the most important lesson isn’t that AI created a new security vulnerability. It’s that AI exposed an existing one.
The system involved was old, under-monitored and carrying risks that already existed. The agent didn’t invent the weakness. It simply found it faster than a human would have and acted within the permissions and connections available to it. That’s a much more useful lesson for technology leaders, security teams and organisations preparing for AI adoption.
AI changes the speed of risk, not the nature of it
When incidents like this occur, it’s tempting to conclude that AI itself is the problem. The reality is more nuanced. The vulnerability already existed. The data already existed. The access path already existed.
What changed was the speed at which those conditions became consequential. An autonomous system was able to interact with an environment, discover an unexpected path and produce an outcome nobody anticipated. The result highlights a challenge every organisation should be asking itself:
If an AI system, autonomous agent, compromised account or malicious actor entered our environment today, how far could it get?
Why agents are different
Generative AI primarily creates information, agentic AI can do something more powerful. Agents can retrieve data, invoke tools, call APIs and take actions on behalf of users. Their risk profile is determined less by the model itself and more by the systems they are connected to and the permissions they inherit.
In guidance released earlier this year, the Australian Signals Directorate and international partners warned about risks including privilege escalation, weaknesses in agent frameworks, cascading failures across connected systems and accountability gaps.
The common thread is clear:
Agent risk is driven by access, permissions and autonomy, not intelligence alone, agents inherit the environment you give them and most organisations already know where their security challenges sit.
They are the same issues security teams have been managing for years:
- Excessive permissions
- Standing administrative access
- Legacy applications
- Weak authentication controls
- Forgotten service accounts
- Poorly protected data
- Over-permissioned SharePoint environments
- Unmonitored applications and APIs
These are not new AI problems
They are existing enterprise security problems that become significantly more visible when agents are introduced. An agent will find oversharing faster than a person can. It may produce a harmful outcome without malicious intent. And it can uncover weaknesses that organisations assumed were low risk simply because nobody had stumbled across them yet.
A warning we were already given
Long before the Medicare incident made headlines, cyber security agencies from Australia, Canada, New Zealand, the United Kingdom and the United States issued a joint warning about frontier AI. Their message was straightforward: AI is accelerating the speed, scale and sophistication of cyber threats, reducing the time between vulnerability discovery and exploitation.
Their recommendations were familiar:
- Reduce attack surface
- Strengthen identity controls
- Patch faster
- Replace or isolate legacy systems
- Maintain incident readiness
The Medicare incident wasn’t an exception; it’s an example of the exact trend they warned organisations to prepare for.
Don’t stop building AI. Fix what’s underneath it.
The answer is not to pause AI initiatives or abandon agent programs.
Identity, permissions, data governance, connected systems and legacy technology are becoming foundational components of AI readiness. Organisations that address those areas will be able to scale AI more confidently. Those that don’t may find that agents expose risks they didn’t realise were already there.
About the author
Ray Reyes is a cyber security specialist at Engage Squared, known for helping organisations cut through complexity and build stronger, more resilient Microsoft security environments. Drawing on deep expertise across identity, data protection, threat detection and governance, Ray brings a practical, trusted perspective to the security challenges created by AI and autonomous agents, helping leaders strengthen the foundations they need to adopt AI with confidence.
Get in touch
