Economic Modelling – how useful is it really?

While some of the criticism is well founded (the BIS shrapnel modelling being a case in point), some is less so.

For example, Richard Denniss from the Australia Institute recently criticised the fact that Treasury’s modelling of the Carbon Pollution Reduction scheme showed that as the carbon price rose, methane emissions from each cow or sheep in Australia would fall steadily (or as Richard puts it so eloquently, as the carbon price rises the fewer the farts). Laughable right?

Well, perhaps not once we dig a little closer.

The assumptions about livestock emissions in the Treasury modelling are derived from scientific research on technologies that reduce emissions from livestock. The research suggests that emissions from livestock can indeed be reduced (by modifying feedstock, introducing supplements and through selective breeding), although this will come at a cost (either through lower productivity or by increasing costs).

The Treasury modelling assumes that as the carbon price rises, it becomes worthwhile for farmers to introduce more costly production methods to reduce emissions from their livestock. Hence less farts as the carbon price rises.

So while the treasury modelling of livestock emissions was a simplification of the likely outcome, it produced results that were broadly sensible (and arguably better than anything else available at the time).

Which brings us to the issue:

If economic modelling is this complicated and there is so much disagreement, shouldn’t we simply do something else?

While it is tempting to put our hands up in despair and simply avoid economic modelling all together, the problem is that the alternative is much worse.

This problem arises because almost all opinions on public policy are formed using some kind of model. It’s just that sometimes these models are not formal, and may exist in a policy maker’s head, or on a scribbled on the back of an envelope, rather than being properly thought through.

Add in the complexity involved in most economic debates, throw in some vested interests, take away transparency and we are left with a recipe for public policy failure.

Formal modelling (when it’s done well) helps to move the debate forward by breaking down complex problems into simple metrics, allows interested stakeholders to understand how the results have been arrived at and provides policy makers with information on the likely winners and losers from change.

So if economic modelling is a necessary evil, how do we move towards a world where we can have some confidence in it?

The Australia institute has made calls for a code of conduct for economic modellers to be mandated to make it easier for the layperson to have more confidence in economic modelling.

The code of conduct currently being floated by the Australia institute is mainly about increasing the transparency of economic modelling and proposes that all modelling should include:

1. a list of key assumptions and provide sensitivity analysis where required
2. an explanation of the choice of model and why it is appropriate
3. a complete set of modelling results (not just the ones that suit)

These three simple things would go a long way to ensuring that users and readers of economic modelling advice have the information and tools to understand what the modelling is saying and how credible it is.

However, adopting even these simple principles, will require effort from both the modelling community and those who purchase and use economic modelling services. There will always be incentives to hide or obfuscate results or assumptions where they highlight uncomfortable truths, and it would be naïve to think that we can easily move to a world where all modelling is open and transparent and never free of bias or political intent.

We can improve the current situation though, by making a concerted effort, where it is easy to do so, to make sure that modelling released into the public domain adheres to principles of greater openness and transparency.

This will mean that public institutions need to become more informed buyers of modelling services, and be clearer at the tender stage the level of detail and transparency required.

Economic modelling will never be perfect

There are always going to be vested interests who will use economic modelling to support their world view or discredit things they don’t like.

This doesn’t mean that we should avoid economic modelling altogether.

Rather, it means that users and procurers of modelling services have a responsibility to be informed enough to understand the debates they are involved in and where possible, provide transparent and robust information into the public domain.

A mandated code of conduct would help, but is not a necessary condition to improving outcomes. The same outcomes could be achieved through a very simple voluntary code, which applies to both procurers and producers of economic modelling services.

[1] Including Saul Estlake and Richard Holden from the Australian Financial Review, and lead economists from the Grattan Institute and the Australia Institute.

[2] See the ABC’s recent article titled “the strange modelling used to justify company tax cuts” for a recent example.