Is the policy going to work? Test it first!

More policies should be pre-tested using behavioural experiments, according to a new review undertaken by the ESRI. The study examined the increased use of experimental pre-tests, both internationally and in Ireland. It concludes that such tests can result in savings for the public finances and for business.

The research found a rapid rise in the number of governments and agencies around the world that are conducting behavioural experiments to inform policy. Increasingly, these experiments take the form of pre-tests of alternative interventions, allowing policymakers to select the option shown to work best – and to avoid potentially expensive mistakes.

International examples include pre-tests of consumer labels designed to provide environmental information about products, regulations to protect online gamblers, and interventions to help households save energy following the installation of smart meters. Some pre-tests have been done in laboratories, some via online experiments, and others through large-scale field trials.

The review concludes that while in many cases the pre-test showed that a policy was likely to be effective, in other cases it recorded no effect. Thus, pre-testing can be used to avoid the introduction of costly changes or regulations that do not, in fact, alter outcomes.

Dr Deirdre Robertson, co-author of the study, said “While it is not possible to pre-test all interventions, in many cases a pre-test can be done relatively quickly and cheaply. The result is that policy can be improved by direct evidence of what works, and what does not.”

Some experimental pre-tests of policy are being undertaken in Ireland too. New regulations for energy pricing were pre-tested for effectiveness in a laboratory study. Behavioural tests have been undertaken in tax administration and health service delivery. The proposed introduction of calorie posting on restaurant menus, part of the action plan to tackle obesity, is also being pre-tested.

Head of the ESRI’s Behavioural Research Unit, Professor Pete Lunn, said “Most interventions and regulations impose costs on the public service or on businesses. Pre-testing allows us to be more confident about the benefits and to choose the intervention most likely to deliver them.”