Identifying international best practice for measuring the impacts of rural and community development spending

Evaluation and monitoring are key tools to ensuring that public monies are being effectively spent and that government-funded programmes, and initiatives, are achieving their stated goals and objectives. This report examines international best practice in evaluation and monitoring in areas relevant for the Department for Rural and Community Department (DRCD). DRCD was established in 2017 and has since led Government responsibility for rural and community development in Ireland.

In 2019, DRCD gross expenditure was €291 million. The Department undertakes a diverse range of activities. However, six key areas of common activity were identified across 20 programmes of expenditure.



These are:

 

(i)           community development;

(ii)          rural development;

(iii)         capital infrastructure;

(iv)         enterprise supports;

(v)          employment supports;

(vi)         provision/development of amenities.

 

In this study, we identify international best practice in evaluating and monitoring expenditures across all these areas and assess the degree to which evaluations in Ireland have been adhering to international standards. The objective of the work is to help further develop the Department’s monitoring and evaluation infrastructure.

The research revealed that the methodological approaches for evaluation of some areas relevant to departmental activities (such as capital infrastructure projects, employment supports and amenities spending) is well developed and relatively straightforward involving well defined empirical estimation techniques. However, measuring the impact of community and rural development programs tends to be much more complex and typically requires the use of mixed methods, which combine data analysis with qualitative approaches, to gain an understanding of programme impacts which are often multidimensional and complex. There is no set toolkit for specific programmes and the methodology adopted should be tailored to both the objectives of the programme and the needs of beneficiaries. In some cases, evaluations could be purely empirical or purely qualitative, while others may involve combinations of counterfactual estimation, monitoring techniques and qualitative assessments. 

Approaches to evaluate, and monitor, public sector programmes in Ireland, in areas relevant to departmental activities, were found to generally meet international best practice standards. However, relative to other countries, we found less evidence of rigorous evaluation of enterprise supports.

Secretary General of DRCD, Kevin McCarthy, said:

“I am delighted to see the publication of this report. The report highlights the important investment and supports being delivered by the Department for our communities. The research conducted under our agreement with the ESRI will enable us to better understand the impact of the work of the Department and inform policy development.”

Author of the report and ESRI Research Officer Dr Adele Whelan, commented:

“DRCD’s proactive approach to meeting the challenge of measuring the impact of interventions across a wide range of activities is extremely welcome and underlines the Department’s commitment to maximising the policy impact of its programmes. However, this research report highlights several complexities in demonstrating a causal link between community development interventions and changes in broad measures of community well-being. There are no off-the-shelf approaches to measuring programme impacts and appropriate planning and investment are necessary to ensure effective evaluation, which, in turn, facilitates a dynamic policy environment that is responsive to change.”