Pitfalls of Participatory Programs: Evidence from a Randomized Evaluation in Education in India - Abhijit Banerjee and Esther Duflo (2008) - Abdul Latif Jameel Poverty Action Lab: Executive Training, Evaluating Social Programs
Download
Read an extract
Themes
Pitfalls of Participatory Programs, evaluating social programs, quasi-experimental method, experimental design, treatment group, comparison group, counterfactual, social differences, economical contexts, Pratham programme, sociological bias
Reader
Abstract
Extract
of 4
Abstract
The "Learning to Read" programme, which aims to improve children's reading skills, has been the subject of intense controversy in the media. For some, it is a success; for others, it is a failure. Drawing on material provided by the lecturer, which sets out four different positions on the policy's impact using various methodologies, this exercise aims to examine the type of impact assessment used in each case, the counterfactual employed, and the problems or limitations of the methodology used.
Extract
[...] Pitfalls of Participatory Programs: Evidence from a Randomized Evaluation in Education in India - Abhijit Banerjee and Esther Duflo (2008) - Abdul Latif Jameel Poverty Action Lab: Executive Training, Evaluating Social Programs Method 1 - News Release: Read India helps children Learn to Read. 1. What type of evaluation does this news release imply ? This evaluation is a pre-post one, meaning that it only focuses on the children actually receiving the programme. It measures how the participants improved after the programme. [...]
[...] The main problem of this type of evaluation is that researchers assume that the non participant are identical to the treatment group. Still, the scientific protocol requires to take into account the exterior factors and the potential differences between the two studied groups. Therefore, the obtained results that prove a weak impact of this method might actually suffer from sociological bias. Indeed, several social differences may actually affect the group who did not participate in the programme, and the evaluators are thus unaware of them. [...]
[...] 2. What represents the counterfactual ? The counterfactual is embodied by children who did not participate in the programme but had data collected before and after the programme was completed on the studied group. According to this letter, had the children not received the programme, their progress would have been significantly less remarkable. 3. What are the problems with this type of evaluation ? The main problem that can be spotted in this case regards the « differences in differences » aspect. [...]
[...] This is the reason why such different - and especially negative ones - results can be obtained according to some methods. This element also throws light on method 2's limits. Method 4 - The numbers don't lie, unless your statisticians are asleep 1. What type of evaluation is this report utilizing ? This report is using the multivariate regression. In this method, children from both groups are compared, but the new aspect is that some of the differences that may change the final results are also taken into account. [...]
[...] This is therefore a more scientific and accurate way of studying children compared to the former methods. According to this report, had the children received the programme, it would not have hurt them ; but on the other hand, receiving the programme did not affect their reading level in a positive way. 3. What are the problems with this type of evaluation? The fact that some factors are excluded because considered as irrelevant to the study or identical between the two groups may affect the global outcome. [...]
