Credibility, Relevance, and Policy Impact in the Evaluation of Adult Basic Skills Programs
Posted by Jessica Gilmour on June 02, 2020 in categoryFacts & ResearchcategoryMember Tips

We are pleased to share the latest issue of ProLiteracy’s research journal Adult Literacy Education: The International Journal of Literacy, Language, and Numeracy. This is a special issue titled “Broadening the Lens on Adult Literacy Education Outcomes,” guest edited by Stephen Reder.

Adult Literacy Education is a peer-reviewed online journal that is published twice per year. The journal features research on adult basic and secondary education and transitions into the workplace and higher education. Research reflects best practices in adult education to inform practitioners, researchers, policy makers, and funders.

We are excited to showcase “Credibility, Relevance, and Policy Impact in the Evaluation of Adult Basic Skills Programs: The Case of the New Opportunities Initiative in Portugal,” by J.D. Carpentieri, University College London, Institute of Education; David Mallows, University College London, Institute of Education; José Pedro Amorim, University of Porto, Faculty of Psychology and Education Sciences, Centre for Research and Intervention in Education, and Paulo Freire Institute of Portugal. An excerpt of the article is highlighted below.  

Adult basic education (ABE) policies aim to help adults improve their literacy, numeracy and information and communications technology skills, as well as their qualifications, often in pursuit of economic gains such as better employment and earnings. The large-scale improvement of skills and qualifications has been referred to as a wicked policy problem, suggesting that it is extremely difficult and perhaps even impossible to achieve success in this policy domain. Evaluations have highlighted these challenges, with many programs showing little or no impact. Between 2006 and 2012, the Portuguese government ran a large-scale adult education program, the New Opportunities Initiative (NOI), which focused on the recognition and validation of adults’ existing skills and the development of literacy and numeracy. The NOI was evaluated twice, in 2009 and in 2012. These two evaluations produced very different findings and outcomes: the first evaluation found the NOI to be a success, and led to continued investment, but the second evaluation reached more negative conclusions and was used as a rationale for de-funding the program. In this article we analyze these two sets of evaluations, investigating the reasons for their starkly different conclusions. We find that, while both evaluations had strengths, they also suffered from serious methodological and/or theoretical weaknesses. These weaknesses are part of a broader pattern of evaluation errors that characterize the field of ABE more generally and which make it more likely that ABE policies will continue to fail. Using the conflicting NOI evaluations as case studies, we offer potential solutions to ABE’s evaluation problem, emphasizing the need to collect long-term longitudinal evidence on the causal mechanisms through which policy goals may be achieved.

Continue Reading


Related Posts

Subscribe to our Blog

Never miss a blog post! Sign up to receive daily, weekly, or monthly email blog notifications.

Sign Up


See the Latest Research

Research - White Paper

Research proves the correlation between obtaining literacy skills and the return on investment related to improving an adult’s life and future.

Learn More