The lawyer is still accountable. The AI system acting on her behalf is not. That gap is no longer theoretical.
After convening the first meeting of the Trust in AI Alliance, it is clear this mismatch is emerging as one of the biggest barriers to enterprise AI deployment.
As AI systems move from answering questions to taking action inside professional workflows, a fundamental mismatch is emerging. Execution shifts to the system. Responsibility still sits with the human.
In agentic systems, that model is being reconfigured, but there is still no clear answer to a critical question: how does a human maintain accountability as more of the work is executed by the system?听听
That question was at the center of the inaugural convening of the Trust in AI Alliance, a group bringing together leaders across model development, infrastructure, and enterprise AI deployment, where participants from OpenAI, Google, Anthropic, AWS, and 成人VR视频 discussed what trustworthy agentic systems require in practice.
A clear theme emerged: AI capability is accelerating faster than accountability.
Most systems today are not designed for that standard.
The Shift No One Is Talking About
In the first wave of AI, the defining question was whether a system could produce a correct answer. That is no longer enough.
As AI systems take on multi-step tasks across real workflows, the question is shifting from accuracy to accountability.
As Michael Gerstenhaber, Vice President of Product Management at Google, said during the discussion: 鈥淒elegating agency to a synthetic agent implies鈥痶rust. The more you delegate, the鈥痬ore you need observability, tracing, and audit. It is not one feature. It is defense鈥痠n鈥痙epth.鈥�
In traditional professional environments, accountability is clear. Humans determine relevance, review source material, verify outputs, and take responsibility for outcomes. In agentic systems, that model is evolving.
Retrieval is automated. Context is lost across steps. Outputs appear grounded in source material without preserving fidelity. Tools execute beyond the user鈥檚 visibility.
As Frank Schilder, Senior Principal Scientist at 成人VR视频, noted: 鈥淲hen we move to an agentic workflow, we automate steps that professionals used to perform manually and that introduces new risks: Context can be silently dropped. Source fidelity can become fragile. Maintaining clear accountability becomes more complex.鈥澨�
These are not edge cases. They are structural risks.听We are automating the work, but not accountability.
If You Can鈥檛 Inspect It, You Can鈥檛 Trust It
In regulated industries, trust has never meant blind confidence. It has always meant the ability to verify. That standard is now colliding with how many AI systems听operate.
Accuracy drives experimentation. Inspection determines adoption.
If a system cannot show its work, it cannot be trusted in high-stakes environments.
As Gayle McElvain, Head of TR Labs at 成人VR视频, put it: 鈥淓rrors create liability. For many professionals, trust means 鈥榯rust but verify.鈥� That means building AI systems where verification is built in.鈥�
Across the discussion, several consistent priorities emerged around what trustworthy systems must provide:听
-
- Step-by-step auditability听
- Traceable reasoning and inspectable tool use听
- Durable logs and process artifacts听
- Clear, persistent provenance听
This is not a feature. It is听infrastructure.
Trust Breaks When Source Integrity Breaks
In knowledge-based professions, trust depends on the integrity of source material.
Agentic systems introduce new failure modes. They may paraphrase where precision is required. They may surface outdated information. They may blur the boundary between authoritative sources and generated reasoning.
These are not cosmetic issues. A single altered word in a statute can change its meaning. A misapplied version of a regulation can create real consequences.
As Zach Brock, Engineering Lead at OpenAI, described: 鈥淲e are moving toward agents that share durable scratch spaces. Citations, version identifiers, and hashes of source material can travel through a workflow without being compressed away.鈥�
That level of persistence is听not a technical听detail. It is what makes accountability possible.
Without it, professionals cannot trace how an answer was constructed or verify whether it reflects the correct source at the correct point in time. Without it, accountability breaks.
Accountability does not听emerge听automatically from more capable systems. It must be explicitly defined.
As Byron听Cook, Director of Automated Reasoning at AWS, said: 鈥淲ith AI, some of those socio-technical mechanisms go away. We听have to听define the dividing line between behaviors we听accept听and those we do not鈥攁nd enforce that symbolically. Without that, accountability cannot be听maintained听as systems take on more of the work.鈥�
This Is a Systems Problem
Much of today鈥檚 AI development is听optimized听for performance benchmarks. But in real-world environments, performance is only part of the equation.
As Scott听White, Head of Product, Enterprise at Anthropic, noted: 鈥淏enchmarks measure whether a model can do the task.听Enterprises are asking a bigger question: will the system around it hold up in the environments where the work听actually happens?听A trustworthy agent听requires听the model, the boundaries around it, and the record of what it did. Getting all three right is what turns AI from a powerful tool into a听system enterprises听can trust with important work.听That’s听what will drive the next wave of adoption.鈥�
Trustworthy systems must be designed to operate safely under pressure, with clear boundaries and strong safeguards.
That requires:听
-
- Clear separation between system instructions and external content听
- Built-in safeguards against prompt injection and data leakage听
- Continuous monitoring and testing听
- Audit trails aligned with regulatory expectations听
Agentic AI is not just a model challenge. It is a governance challenge.
The Next Phase of AI
We are entering a new phase of AI adoption, one defined not by experimentation, but by deployment inside real workflows.
The industry is shifting from outputs to systems, from benchmarks to reliability, and from capability to accountability.
But this shift will not happen automatically. It requires new standards for auditability, clearer approaches to provenance, and systems designed to preserve truth and responsibility across every step of a workflow. These are solvable problems鈥攂ut only if accountability is designed into the system from the start.
The organizations that solve this will define the next generation of AI.
In high-stakes domains, trust is not optional.
It is not a feature. It is the product.
The Trust in AI Alliance was announced in January to bring together leaders across the AI ecosystem to advance practical standards for accountability, transparency, and trust in AI systems. The group will continue to meet regularly, with听select听insights from those discussions shared publicly.听
