Automating The Big State Will Need More Than Computers

Robodebt — the automated Centrelink debt issuance program that was found invalid by a federal court last month — is not just an embarrassment for the government.  It is the first truly twenty-first century administrative policy debacle.

Australian governments and regulators increasingly want to automate public administrative processes and regulatory compliance, taking advantage of new generations of technologies like artificial intelligence and blockchain to provide better services and controls with lower bureaucratic costs.  There are good reasons for this.  But our would-be reformers will need to study how robodebt went wrong if they want to get automation right.

The robodebt program (officially described as a new online compliance intervention system) was established in 2016 to automate the monitoring and enforcement of welfare fraud.  Robodebt compared an individual's historical Centrelink payments with their averaged historical income (according to tax returns held by the Australian Taxation Office).  If the Centrelink recipient had earned more money than they were entitled to under Centrelink rules, then the system automatically issued a debt notice.

That was how it was supposed to work.  In practice robodebt was poorly designed, sending out notices when no debt actually existed.  Around 20 per cent of debts issued were eventually waived or reduced.  The fact that those who bore the brunt of these errors had limited financial resources to contest their debts contributed to robodebt's cruelty.  In November, the federal court declared that debts calculated using the income average approach had not been validly made, and the government has now abandoned the approach.

Automation in government has a lot of promise, and a lot of advocates.  Urban planners are increasingly using AI to predict and affect transport flows.  The Australian Senate is inquiring into the use of technology for regulatory compliance ("regtech") particularly in the finance sector.  Some regulatory frameworks are so byzantine that regulated firms have to use frontier technologies just to meet bare compliance rules:  Australia's adoption of the Basel II capital accords led to major changes in IT systems.  And the open banking standards being developed by CSIRO's Data61 promise deeper technological integration between private and public sectors.

Regulatory compliance costs can be incredibly high.  I have estimated that red tape costs the economy around 11 per cent of GDP in foregone output.  The cost of public administration to the taxpayer is considerably more.  Anything that lowers these costs is desirable.

But robodebt shows us how attempts to reduce the cost of administration and regulatory compliance can be harmful when done incompetently.  The reason is built into the modern philosophy of government.

Economists distinguish between administrative regimes governed by discretion and those governed by rules.  The prototypical example here is monetary policy.  Rules-based monetary policies, where central banks are required to meet targets fixed in advance, are less flexible (as the RBA, which has consistently failed to meet its inflation target is keenly aware) but at the same time provide a lot more certainty to the economy.  And while discretionary regimes are flexible, they also vest a lot of power in unelected bureaucrats and regulators, which comes at the cost of democratic legitimacy.

Automation in government is possible when we have clear rules that can be automated.  If we are going to build administrative and compliance processes into code, we need to be very specific about what those processes actually are.  But since the sharp growth of the regulatory state in the 1980s governments have increasingly relied less on rules and more on discretion.  ASIC's shrinks-in-the-boardroom approach to corporate governance is almost a parody of the discretionary style.

The program of automating public administration is therefore a massive task of converting — or at least adapting — decades of built up discretionary systems into rules-based ones.  This was where robodebt fell over.  Before robodebt, individual human bureaucrats had to manually process welfare compliance, which gave them some discretion to second-guess whether debt notices should be sent.  Automating the process removed that discretion.

The move from discretion to rules is, to be clear, a task very much worth doing.  Discretionary administration feeds economic uncertainty, and ultimately lowers economic growth.  We have a historically unique opportunity to reduce the regulatory burden and reassert democratic control over the non-democratic regulatory empires that have been building up.

Of course, public administration-by-algorithm is only as effective (or fair, or just, or efficient) as those who write the algorithm build it to be.  There's a lot of discussion at the moment in technology circles about AI bias.  But biased or counterproductive administrative systems are not a new problem.  Even the best-intentioned regulations can be harmful if poorly designed, or if bureaucrats decide to use discretion in their interest rather than the public interest.

Robodebt failed because of an incompetent attempt to change a discretionary system to a rules-based system, which was then compounded by political disregard for the effect of policy on welfare recipients.  But robodebt is also a warning for the rest of government.  The benefits of technology for public administration won't be quickly or easily realised.

Because when we talk about public sector automation, we're not just talking about a technical upgrade.  We're talking about an overhaul of the regulatory state itself.