When the inquiry into automation relied on automation – and proved why human oversight still matters.
Summary:
The Robodebt scheme became one of Australia’s most notorious examples of automation gone wrong. Thousands of people were significantly damaged by this failure to supervise and use technology appropriately. Now, in an extraordinary twist, a taxpayer-funded review of Robodebt has itself fallen victim to the same problem, this time through the use of artificial intelligence by Deloitte. The episode offers a timely reminder that technology, no matter how sophisticated, cannot replace human responsibility.
A Cautionary Tale
Robodebt was built on automation. By matching tax data against Centrelink payments, the scheme issued debt notices based on income averaging rather than actual earnings. Thousands were accused of owing money they didn’t owe. The courts later ruled the debts unlawful, and a Royal Commission exposed the program’s legal and moral failures.
In 2025, Deloitte was engaged by the Australian Government to review aspects of the post-Robodebt compliance framework. To assist in preparing the report, the firm used artificial intelligence. The resulting document included serious factual errors, from fabricated academic references to misquoted court judgments. Deloitte has since apologised, refunded $98,000 of the $440,000 contract, and released a corrected version.
It was, quite literally, automation reviewing automation and failing in the same way.
A Few Lines from “Ironic” that Hit the Mark
Alanis Morissette’s 1995 classic suddenly feels like a Robodebt anthem.
“An old man turned ninety-eight, he won the lottery and died the next day.”
The government thought it had struck gold with AI-powered efficiency, only to see credibility collapse overnight.
“It’s a black fly in your Chardonnay.”
The fly was a fabricated legal citation a small detail that spoiled the whole glass.
“It’s meeting the man of my dreams and then meeting his beautiful wife.”
A reminder that prestige and expertise mean little if the process behind them is flawed.
“Well, life has a funny way of sneaking up on you.”
Who would have guessed that the review of a failed automated system would stumble over the same kind of automation?
How the Errors Happened
The AI tools used in preparing the report generated “hallucinations”, information that sounded legitimate but was entirely made up. The report contained invented academic articles, incorrect legal quotes, and false attributions.
While Deloitte maintained that the core conclusions were unaffected, the incident has reignited debate about transparency, accountability, and the professional duty to verify AI-assisted work before publication.
The symbolism is unmistakable. Robodebt failed because it replaced judgment with data. The report that analysed it faltered because it replaced diligence with technology.
Key Lessons for Clients and Professionals
- Human responsibility remains non-negotiable
Whether drafting legal opinions, compliance reports, or policy documents, AI is a tool, not a substitute for judgment. The author or firm remains accountable for every word.
- Governance matters as much as innovation
Firms must implement clear AI governance: disclose when AI tools are used, document their role, and verify outputs through rigorous review.
- Protect credibility at all costs
A single error in an official or public document can erode trust built over decades. Reputation, once damaged, is difficult to repair.
- Beware of “automation blindness”
The more confident a process appears, the easier it is to miss errors. The Robodebt story and its ironic sequel show that trust in technology should always be balanced with scepticism.
Irony Comes Full Circle
Robodebt began as an attempt to make human welfare processing more efficient through automation. It ended in legal invalidity, human harm, and public outrage. Now, in seeking to understand those mistakes, a new report has made its own through AI.
If Alanis Morissette were writing today, she might add a verse just for Canberra:
“It’s a faulty report by AI, when the whole point was to expose automation’s lie.”
The message for lawyers, regulators, and policy advisers is simple technology can assist, but it cannot absolve. The law depends on careful human judgment, empathy, and accountability. Without those, irony becomes tragedy.
Key Takeaway
Even the best technology cannot fix a failure of human oversight. In the era of AI-driven decision making, lawyers and advisors must ensure that responsibility remains human and that automation never replaces accountability.
Please do not hesitate to contact us through our website www.lynnandbrown.com.au or call us on (08) 9375 3541 to make an appointment to discuss the above.
About the Author: This article is authored by Steven Brown. Steven Brown’s legal career covers working with multinational corporations and Australian listed companies to family-owned businesses. This range of experience has equipped Steven with the unique ability to offer tailored legal services that make a significant difference to businesses of all sizes.
