Some simple ways to better understand economic modelling

To follow up on this theme, we’ve put together five simple tips that can help you to become a more informed reader of economic modelling.

While there is no substitute for expert advice, there are some simple things to keep in mind that will go a long way to helping you make an informed assessment of an economic modelling report.

It is unusual for economic impacts to be multiples of the direct impacts. These generally only occur where extreme economic conditions prevail (such as where there are absolute rigidities in markets or very high levels of unemployment). These extreme economic conditions have not been observed in Australia for decades.  For this reason it should be unusual for models to produce net economic impacts that are multiples of the direct impacts.[1] Where they do, it is often worth questioning if the assumptions underpinning the results are valid.

Often, the modelling results can be tested using simple calculations.

Although we don’t want to enter into the debate about the pros and cons of changes to negative gearing, the BIS shrapnel report released on this issue provides a very good case in point. As was pointed out by John Daly from the Grattan Institute[2], the modelling in this report had a number of glaring inconsistencies that should have rung alarm bells for readers.

The most obvious of these was the suggestion that the estimated 7,200 fewer homes that would be built would reduce GDP by $19 billion. A simple calculation shows that this implies that every additional house or apartment constructed in Australia would increase the size of the economy by $2.6 million, much more than could reasonably be expected. [3]

There can be very good reasons why modelling results are much larger (or smaller) than expected. However, unless the reasons for this are clearly spelt out, the reader would do best to treat the results with suspicion.

Most discussion around economics is concerned with jobs and growth (as measured by gross domestic product or GDP). However, for many policy options these are the wrong measures, and can provide a misleading picture of the veracity of the benefits or costs of policy change.

This is because employment and GDP are measures of production (how hard we work) rather than a measure of the benefits we receive from this work (income).

Take emissions trading policy. Under most modelled scenarios with global trading, Australia would become a net purchasers of emissions permits from overseas (since we have relatively high emissions intensities due our reliance on abundant reserves of coal and gas). These foreign permit purchases deduct from the incomes earned in Australia, but are not counted in GDP. That means that emissions trading policies might have only a very small impact on GDP, but have a much bigger impact on incomes.

Better measures of welfare are real Gross National Product (GNP) as opposed to gross domestic product (the most frequently quoted parameter), which accounts for income flows to foreigners and real Gross National Income (GNI), which accounts for both foreign incomes flows and differences between the prices of goods consumed and goods produced (important when policies impact on prices[4]).

Often the key drivers of change will be hidden deep in a modelling report. This means that most often, it is not possible to understand the results simply by reading the executive summary of a report or its’ associated press release.

Often a reading of the modelling detail will throw up an obvious flaw or identify a key assumption that has been used to drive results. Most credible modelling will discuss uncertainties and key assumptions upfront and may even decompose the results into the key factors driving outcomes. Where this detail isn’t available, treat the results with caution or seek out answers.

All modelling is a simplification – after all the real world is a complex place, and even very detailed models are not capable of replicating all of this complexity.

This means that different models may give different answers to the same question. While this may seem a great weakness – it is actually a strength. Rather than showing that the modellers don’t know what they are talking about, different results may reflect that we simply don’t know precisely what the impacts of a policy change will be, or there is significant uncertainty around the key parameters that influence results. There is often value in understanding these differences as they can illuminate the contentious issues under consideration.

Credible modelling makes these uncertainties transparent and run sensitivity tests to demonstrate how varying these parameters might affect the results.

Where this analysis is missing or the uncertainties faced by the modeller are not clear, question whether you have the full picture.

These include:

• The economy operates within constraints. At the very least economic modelling should consider the limited supplies of skilled workers and equipment, but might also consider how markets are constrained by transport networks or how migration is constrained by preferences.
• The economy is dynamic. Firms and workers respond to incentives (such as prices and wages) and the market usually finds solutions to complex problems.
• Government expenditures have to be paid for. Modelling should fully account for this and avoid the modelling trick of assuming that governments are able to borrow with no cost to taxpayers.

These realities mean that it is not possible to generate long lasting employment gains by simply spending more money. It also means that the net economic impacts of Government spending will often less than the direct expenditures (sometimes significantly so) – so that a million dollars of expenditure may very well generate less than one million dollars of economic activity (implying there should be a non-economic rationale for the expenditure). A good example of this would be the Rudd Government’s insulation program.

Finally, remember that it is an economist’s job to explain the results of their complex models in a way that people can understand. As Einstein once said if you can’t explain it to a six year old, you clearly don’t understand it yourself. While many economic issues are complex, there is never a reason why they can’t be clearly explained to an informed adult.

[1]     Although there is no hard and fast rule, be suspicious if the net impact is more than 1.5-2 times the direct impact.

[2]     https://grattan.edu.au/news/why-it-is-good-policy-not-bad-politics-to-ignore-bad-modelling-on-negative-gearing/

[3]     The contribution to GDP from constructing a house should be less than the cost of construction since any imported components (including those along the supply chain) should be netted out.

[4]     This is important when considering trade policies, such as subsidies to exports. These subsidies might increase production, but are unlikely to result in an increase in real income since they cause export prices to fall at the expense of production for domestic use.