When AI Stops Recommending and Starts Committing

For three years, most Australian businesses have used AI the same way: ask it something, read what comes back, decide what to do. The output was a draft, a summary, a suggestion. If it was wrong, you noticed, and you moved on.

That relationship is changing. AI systems can now use tools, call APIs, read your systems and take actions in them. They can research a supplier, build a purchase order, submit it, and follow up when it isn't fulfilled. The industry calls these agents, and the important thing about them has nothing to do with how clever the underlying model is.

A poor chatbot answer wastes a few minutes. A poor agent action spends money, discloses information, accepts terms, or creates an obligation in your name.

The difference is between a recommendation and a commitment. You can throw a recommendation away. A commitment has a counterparty.

This is not a 2028 problem

In January 2026, Mastercard ran a controlled, authenticated agentic payment trial in Australia with Commonwealth Bank and Westpac — the second published market outside the United States, after the UAE in November 2025. That was a controlled trial, not general availability, and the distinction matters. But it establishes something that was previously theoretical: an AI agent completed an authenticated payment with an Australian bank.

Meanwhile the plumbing has been arriving quickly. There are now open standards for how a model reaches your tools, how agents coordinate with each other, how a purchase gets discovered and checked out, and — most recently — how the authority a human granted to an agent can be recorded as verifiable evidence.

Notice the order. The capability layers came first and arrived fast, because they were built by engineers optimising for capability. The trust layer underneath them — who this agent is, what it was allowed to do, and how you prove it afterwards — is being built later, deliberately, by different people. That gap is where the risk currently sits, and it is not a criticism of anyone. It simply wasn't what the first wave was solving for.

Five questions, before anything else

When an agentic deployment goes wrong, the failure is rarely that the model produced a stupid answer. It is that the system around the model had no answer to one of these five questions. We use them as the frame for every agentic engagement.

1. Identity

Who is this agent, who operates it, and who does it represent? Not “which model is it” — which legal entity stands behind it, and can you tell it apart from something pretending to be it?

Fails as: impersonation. Agent traffic is already being spoofed at scale in production web traffic, and detection vendors now track it as a category of its own.

2. Authority

What has it actually been authorised to do? This is the distinction most organisations collapse: authentication is not authorisation. Proving who is acting tells you nothing about what they were permitted to do.

Fails as: a valid signature on a transaction nobody ever approved. The signature is real. The authority behind it was never established.

3. Constraints

Spending caps, approved suppliers, prohibited actions, thresholds that force a human decision. Critically, these have to be enforced outside the model, by a service that cannot be argued with.

Fails as: two agents drawing on the same budget in parallel and both being told yes. Instructions in a prompt are not a control. A reservation against a shared ledger is.

4. Evidence

Can you reconstruct what the agent was asked to do, what it did, and the basis on which it acted — months later, to someone who wasn't there?

Fails as: a supplier invoice arrives with altered bank details, the agent pays it, and nobody can demonstrate what the original approval was bound to.

5. Recourse

When something goes wrong, can you correct it, compensate the customer where appropriate, and determine who is accountable?

Fails as: discovering that stopping the agent and undoing what it did are two very different capabilities. A global pause halts new commitments. It cannot reverse a settled transfer.

We describe these as governance because that is the language boards and regulators use. From a customer's side of the counter, they are something simpler. Nobody has ever chosen a supplier because of its governance framework. They care whether the system did what it was supposed to do, and whether somebody will fix it when it doesn't.

Consent, accountability, evidence and redress are what people already expect from commerce. Agentic systems don't get an exemption from them.

Where Australian law actually sits

There is no Australian AI Act, and the position is more settled than the commentary suggests. Existing obligations — financial services, credit, consumer, privacy, sanctions, AML/CTF — already apply to what an agent does on your behalf. What has changed recently is narrower and more dated than most summaries allow.

• Now — The National AI Centre's six Essential AI Practices are the operative voluntary guidance. They superseded the earlier ten-guardrail Voluntary AI Safety Standard, which is worth knowing, because a good deal of advice still circulating cites the old ten.

• 30 April 2026 — APRA's letter to industry is, as far as we can establish, the only written statement by an Australian regulator that explicitly addresses autonomous agents and agentic workflows, warning that assurance practices are failing to keep pace. ASIC's May letter, sometimes cited alongside it, is about cyber resilience and does not use the term.

• 10 December 2026 — New automated decision-making transparency obligations commence under the Privacy Act. Be precise about what they are: a privacy-policy disclosure duty covering the kinds of decisions made and the kinds of personal information used. They do not create a right to an explanation of a specific decision, or a right to human review of one.

• April 2027 — The Digital Assets Framework amendments commence, creating two new licensed financial product categories. A six-month transition runs from commencement, not from Royal Assent.

• 1 July 2027 — New unfair trading practices provisions commence, capable of capturing manipulative AI conduct and algorithmic dark patterns where the statutory elements are made out. Note the carve-outs for body-corporate consumers and business supplies.

On consumer law more broadly, Treasury's 2025 review found the Australian Consumer Law broadly capable of dealing with AI-enabled goods and services, while identifying specific areas for further work. That is a more careful finding than “fit for purpose,” and the gap between those two phrasings is roughly where the practical questions live. The durable point is the one businesses most often get wrong: the fact that an AI tool made the decision does not shield the business from liability for it.

How to start, without stopping

The instinct in most organisations is either to ban agentic tooling outright or to let it run and hope. Neither survives contact with a board.

The workable approach is progressive, and it looks a lot like how you already handle people. A new employee gets a defined role, explicit delegations and supervision. Their authority grows as trust is established. With software, most of those controls can be made explicit and machine-enforceable — which is an advantage, not a burden.

In practice that means moving through stages rather than switching something on: read-only first, then the agent proposing actions a human approves, then human-approved execution, then bounded autonomous execution inside limits you have tested. And expanding those limits along separate axes — by transaction size, by counterparty, by how reversible the failure would be — rather than all at once.

Start with one workflow where the inputs are structured, the outcome is checkable, the exposure is capped, and someone owns the exceptions. Instrument it properly. Then look at what the evidence actually shows before you widen the mandate.

Autonomy is not a switch. It is something you delegate progressively, and the evidence is what earns the next increment.

The question worth asking of any agentic deployment is not whether the technology is capable of the task. It almost certainly is. The question is whether you can show what it was authorised to do, prove what it did, and put things right when it gets one wrong.

Working through this

ValiDATA AI designs AI management systems aligned to ISO/IEC 42001 — adopted in Australia as AS ISO/IEC 42001:2023 — for organisations in regulated industries. We work through the five questions above against a real workflow, and produce the responsibility map, control specification, evidence design and incident playbook that go with it.

If you are looking at delegating authority to an agent and want to know what has to exist first, get in touch.