What is needed to keep increasingly capable AI systems safe and accountable?
By Kinza T. & Zarnab Fayyaz
Recent disclosures from Anthropic have added to wider concerns about the pace and capabilities of AI.
The company’s reported IPO filing pointed to risks such as models resisting shutdown, manipulating information, and behaving in ways that could create accountability challenges. As AI systems become more autonomous, capable of performing tasks, and connected to real-world systems, questions around oversight, accountability, and control are becoming increasingly important.
In this third edition of Expert Room, Green Minds explores what stronger AI accountability could look like as systems become more capable, autonomous, and embedded in real-world decision-making.
“AI Requires Continuous, Risk-Based Governance”
Raphael Mendes Motta
Business Engineering & AI Advisor, Oracle
Raphael Mendes Motta argues that AI governance should develop alongside model capabilities, with continuous risk assessment, independent evaluation, and meaningful human oversight built into how systems are deployed and monitored.
“More capable AI requires continuous, risk-based governance, independent evaluations, and human oversight to intervene. We can delegate tasks to AI, but not responsibility for their consequences.”
His perspective reinforces a core principle of AI accountability: organizations may delegate tasks to AI systems, but they cannot delegate responsibility for the outcomes those systems create.
“AI Safety Needs Strong Governance and Human Oversight”
Cosme Garcia
Sr. Technical Program Manager, Data Center AI Platforms Development, Google
Cosme Garcia emphasizes that autonomous AI systems require governance disciplines similar to those used for human decision-making, especially where access, oversight, auditability, and intervention are needed.
“AI safety and accountability should follow the same frameworks as you apply to humans because of the nature of agent autonomy. Strong governance, least privilege, compartmentalization (sandboxing), and human-in-the-loop, audit and monitoring practices are essential.”
This perspective highlights the importance of strong governance, least-privilege access, sandboxing, human oversight, and continuous audit and monitoring as AI systems become more autonomous.
“AI Safety Needs to Be Treated Like Cybersecurity”
Associate Director – Technology at a leading global telecommunications company
In this view, AI governance can no longer rely only on static rules. As systems become more agentic, organizations need oversight that operates continuously while the system is running.
“As AI systems become more capable and agentic, governance needs to move from static policy to continuous, runtime oversight.”
The cybersecurity comparison reframes AI safety as an operational discipline: organizations should assume failures will occur, test for them continuously, limit their potential impact, and create mechanisms to detect, contain, and learn from incidents.
“The biggest shift is treating AI safety like cybersecurity: assume failures will occur, continuously test for them, limit the potential blast radius, and have clear mechanisms to detect, contain, and learn from incidents.”
The perspective highlights the need for continuous monitoring, controlled access, auditable decision trails, and clear human intervention points for high-risk actions.
The Green Minds Take
Across the three perspectives, one message stands out: AI safety needs to evolve as quickly as AI capabilities do.
From continuous risk monitoring and real-time oversight to controlled access, human intervention, and clear accountability, the experts point to the need for stronger safeguards as AI systems become more autonomous.
The takeaway? AI can move fast, but governance needs to keep pace.