No Evidence that the Back To Education Allowance employment support programme is effective in helping unemployed people to find jobs says the ESRI

No evidence of improved employment outcomes for Back To Education Allowance recipients, says the ESRI.  The main objective of the Back To Education Allowance (BTEA) employment support scheme is to raise the education and skill levels of unemployed people in order to help them to progress into employment [1]. However, a new report, commissioned by the Department of Social Protection (DSP) and published today (Tuesday, 3 November 2015) by the Economic and Social Research Institute (ESRI), has found no evidence that people who commenced an education programme in 2008 under the BTEA scheme benefitted from improved employment outcomes in the period up to 2014. In fact, compared to similarly unemployed individuals, jobseekers who commenced an education course supported by the BTEA programme in September/October 2008 were between 23 and 38 percentage points less likely to have exited unemployment to a job by June 2012, and between 14 and 29 percentage points less likely at the same time in 2014. The results from the report, titled An Evaluation of the Back to Education Allowance, by Elish Kelly, Seamus McGuinness and John Walsh (ESRI), were presented at an event for public officials, academics and journalists this morning. The Tánaiste and Minister for Social Protection, Joan Burton TD welcomed the report and thanked the ESRI for undertaking the evaluation on behalf of the Department, noting: “The Department of Social Protection has a long track record of commissioning independent evaluations of the schemes that it operates and in then using the results to inform future policy direction. We have done this in the past with an evaluation of the National Employment Action Plan being used to inform the development of Intreo. Similarly an evaluation of JobBridge, although it showed very positive outcomes, still informed some changes made to the detail of the scheme. Likewise the results published today will be taken into account in considering how we can change the BTEA scheme to deliver better outcomes.” The Tánaiste and Minister for Social Protection went on to say: “In this regard, the Department has already made a number of changes to the BTEA scheme, including vetting applications to assess their labour market relevance.  It has also asked an expert group — the Labour Market Council — for its recommendations as to how to adapt the BTEA scheme in the light of the ESRI study results. The Labour Market Council has made six recommendations, all of which are being acted on. These include, for example, a recommendation to commission a qualitative evaluation of BTEA participants and practitioners to explore the factors giving rise to the results, and a recommendation to develop linkages between Department of Social Protection and Education sector IT systems to enable better tracking of BTEA participant progress. In concluding, the Tánaiste and Minister for Social Protection said: “Our swift response to these recommendations demonstrates my commitment and that of the Department to ensuring that future policy developments are truly evidence-ased and informed not just by the data but by meaningful insights as to what is driving that data.” What is the BTEA Scheme? The BTEA scheme, which was established in 1998, is a non-statutory, second-chance education scheme for jobseekers, lone parents and people with disabilities who would like to undertake a full-time education course. It consists of two study options — the Second-Level Option (SLO), which predominantly covers Junior Certificate to Post Leaving Certificate courses; and the Third-Level Option (TLO), which funds undergraduate and certain postgraduate courses. In the case of jobseekers, who are the focus of the study published today, the BTEA scheme allows recipients to participate in a course of education, while unemployed, and continue to receive an income support equivalent to their jobseeker payment. As part of the BTEA scheme, they are excused from the requirement to be actively seeking work for the duration of their education course. BTEA Scheme Expenditure Between 2007 and 2012, the DSP’s spending on the BTEA more than trebled, increasing from €64.1 million to €199.5 million per annum. Over the same time period, the number of recipients quadrupled, growing from approximately 6,000 to almost 25,000 per annum. This increased spending on the BTEA scheme by the DSP, and growth in recipient numbers, was as a result of the unemployment crisis that evolved from the 2008 Great Recession which resulted in the unemployment rate increasing to a peak of 15.1% in early 2012.  Expenditure on the BTEA scheme has since decreased, falling to €162.5 million in 2014, as has the number of people on the scheme (22,700 in 2014). Nevertheless, after the Community Employment (CE) scheme the BTEA represents the second largest activation measure in Ireland, in terms of both expenditure and scheme participants (see Notes for Editor for BTEA Recipient Numbers and Expenditure).  How Effective is the BTEA Scheme in Assisting Unemployed People to Progress to another Education or Training Course? In addition to gaining employment, the report published today also examined how effective the BTEA scheme is in facilitating progression to another education programme, training course or employment placement. In this regard, the authors found that both the second-level and third-level BTEA course options increased a jobseeker’s chances of being in education or training between four and six years following entry into their original BTEA courses. However, the numbers pursuing this path were quite small. Furthermore, over half of those who continued to be in education were undertaking a similar course of study to the programme that they had entered into in 2008, as opposed to progressing into a more advanced education or training programme. Further Research The authors stress that the data used to conduct the BTEA evaluation presented in the report do not contain any information that casts light on why the results derived on the BTEA scheme’s effectiveness are as they are, or on the individual experiences or processes that contributed to the observed results. Therefore, conclusions cannot be drawn with respect to the quality or impact of the educational programmes undertaken by BTEA claimants. In fact, the authors indicate that for comprehensive policy conclusions to be drawn regarding the factors underling the BTEA scheme’s effectiveness, the evaluation conducted by them would need to be complemented by qualitative research that collected detailed evidence from BTEA participants, education providers, employers of BTEA completers and officials from both the DSP and the Department of Education and Skills (DES). Commenting on the report, Dr. Elish Kelly, lead author and research officer at the ESRI, said: “The analysis conducted in this report supports a move to the implementation in Ireland of international best practice in evaluating how effective Government programmes are in assisting unemployed people to find jobs. In particular, policymakers need to move away from using summary statistics when examining a scheme’s effectiveness. Instead they need to focus on using, what are known internationally as, ‘counterfactual methodological techniques’ to identify if the programmes that they are spending money on are being successful in achieving their goals. The creation of the Jobseeker Longitudinal dataset by the Department is a positive step in this direction, as the findings from this study indicate that the Department’s new dataset provides a robust basis for quantitative analysis of the effectiveness of the Department’s employment support schemes using counterfactual techniques”. Specifically in relation to the report’s findings on the BTEA scheme’s effectiveness, Dr. Kelly said: “There is little doubt that schemes that support access to education, like the BTEA, are a vital component of any life-long learning strategy. However, the evidence presented in this report, which is consistent with the findings from past reviews of the programme, raises concerns about the effectiveness of the BTEA scheme as an employment support programme for assisting jobseekers to transition from unemployment to employment.” “While there is some evidence that the BTEA scheme was successful in redirecting participants to further study or training, the programme does not appear to be effective in terms of its core public policy objective  of assisting the unemployed to transition to employment.” Dr Kelly went on to say: “Furthermore, we are concerned about the degree of progression into higher level study for those BTEA participants that go this route of continued education. In 2012 and 2014, many of these individuals appeared to have been transitioning into programmes that were at the same level as the courses that they entered into when they first commenced a BTEA course in 2008, as opposed to them having moved on to a more advanced education or training programme”.

Note for Editors on Intreo: Intreo is a new service from the Department of Social Protection. Specifically, it is a single point of contact for all employment and income supports. The new service, which was initially launched in October 2012 and then rolled-out country-wide over the next two years, was designed to provide a more streamlined approach. Intreo offers practical, tailored employment services and supports for jobseekers and employers alike (for more details, see http://www.welfare.ie Note for Editors on BTEA Scheme Recipient Numbers and Expenditure: BTEA Recipient Numbers and Expenditure: 1998 - 2014(1)

Source:    Statistical Information on Social Welfare Services (Departmental Publications, 1998-2014). Notes: (1) Expenditure based on calendar year and recipient numbers based on 31 December of each year. (2) Provisional. Note for Editors on the Jobseekers’ Longitudinal Dataset (JLD): The study published today was carried out using anonymised data from the DSP’s newly constructed Jobseeker Longitudinal Dataset (JLD). This dataset, which was created through the amalgamation of five administrative data sources, tracks the social welfare claim, employment, training and activation programme experiences of jobseeker claimants since 2004. In terms of evidence-based policy, the development of the JLD represents a significant step forward in Ireland as the dataset allows for rigorous evaluation of activation programmes like the BTEA. The evaluation of the BTEA scheme was developed as a ‘pathfinder’ study for the purposes of understanding how the dataset might be utilised to best effect. The DSP is now making the dataset available for research purposes to accredited researchers under the Statistics Act and is itself, together with the Irish Government Economic Evaluation Service (IGEES) using the dataset to conduct/commission further evaluations of scheme and programme effectiveness. Schemes to be evaluated in the coming months include JobBridge, Intreo process reforms and the Back to Work Enterprise Allowance.

[1] Department of Social and Family Affairs (2005). Report of the Working Group on the Review of the Back to Education Allowance (BTEA) Scheme. Dublin: Department of Social and Family Affairs. ENDS.