BMJ, published in the UK, is a premier medical journal, and not a t...
Esther Duflo and Abhijit Banerjee shared the 2019 Nobel prize in Ec...
Randomized controlled trials became a popular tool for policy evalu...
These numbers have not improved significantly since this was publis...
This programs are what they sound like: the treatment is to give ca...
Banerjee and Duflo are both part of JPAL (The Abdul Latif Jameel Po...
The incentives (lentils in this case) are a key treatment that they...
With any survey, particularly health-related surveys, there are man...
Clustered randomized controlled trials are different than individua...
This is key. If the measurement error was correlated with the treat...
Many of the RCTs for development are longitudinal studies that take...
This is a key visual presentation of the results. Note that the ...
The program with incentives pays for itself on a per child basis.
The ultimate goal of JPAL and development economics is to evaluate ...
RESEARCH
Improving immunisation coverage in rural India: clustered
randomised controlled evaluation of immunisation
campaigns with and without incentives
Abhijit Vinayak Banerjee, Ford foundation international professor of economics,
1
Esther Duflo, Abdul Latif
Jameel professor of poverty alleviation and development economics,
1
Rachel Glennerster, executive
director,
2
Dhruva Kothari, medical student
3
ABSTRACT
Objective To assess the efficacy of modest non-financial
incentives on immunisation rates in children aged 1-3 and
to compare it with the effect of only improving the
reliability of the supply of services.
Design Clustered randomised controlled study.
Setting Rural Rajasthan, India.
Participants 1640 children aged 1-3 at end point.
Interventions 134 villages were randomised to one of
three groups: a once monthly reliable immunisation camp
(intervention A; 379 children from 30 villages); a once
monthly reliable immunisation camp with small
incentives (raw lentils and metal plates for completed
immunisation; intervention B; 382 children from 30
villages), or control (no intervention, 860 children in 74
villages). Surveys were undertaken in randomly selected
households at baseline and about 18 months after the
interventions started (end point).
Main outcome measures Proportion of children aged 1-3
at the end point who were partially or fully immunised.
Results Among children aged 1-3 in the end point survey,
rates of full immunisation were 39% (148/382, 95%
confidence interval 30% to 47%) for intervention B
villages (reliable immunisation with incentives), 18%
(68/379, 11% to 23%) for intervention A villages (reliable
immunisation without incentives), and 6% (50/860, 3%
to 9%) for control villages. The relative risk of complete
immunisation for intervention B versus control was 6.7
(4.5 to 8.8) and for intervention B versus intervention A
was 2.2 (1.5 to 2.8). Children in areas neighbouring
intervention B villages were also more likely to be fully
immunised than those from areas neighbouring
intervention A villages (1.9, 1.1 to 2.8). The average cost
per immunisation was $28 (1102 rupees, about
£
16 or
19) in intervention A and $56 (2202 rupees) in
intervention B.
Conclusions Improving reliability of services improves
immunisation rates, but the effect remains modest. Small
incentives have large positive impacts on the uptake of
immunisation services in resource poor areas and are
more cost effective than purely improving supply.
Trial registration IRSCTN87759937.
INTRODUCTION
Immunisation is a highly cost effective way of improv-
ing survival in children in developing countries.
12
Every year throughout the world, however, an esti-
mated 27 million children and 40 million pregnant
women do not receive the basic package of immunisa-
tions(asdefinedby WHO andUnicef), and two to three
million people die from diseases that can be prevented
with vaccines.
13
Immunisation rates are in part based
on official statistics and might be over-reported.
45
In
India, immunisation services are offered free in public
health facilities, but, despite rapid increases, the immu-
nisation rate remains low in some areas. According to
the National Family Health survey (NFHS-3), in India
only 44% of children aged 1-2 years have received the
basic package.
6
That drops to 22% in rural Rajasthan,
the setting of the present study, and was less than 2% in
our study area (a disadvantaged population in rural
Udaipur) at baseline (according to a more detailed sur-
vey instrument than the NFHS-3, that was less likely to
overestimate full immunisation rates). Understanding
how to improve the uptake of immunisation is impor-
tant, not only for existing vaccines but also to maximise
the uptake of any new vaccine.
Reliable supply of free immunisation services and
incentives to improve the demand for these services
could improve immunisation rates. Previous studies
have assessed the effectiveness of financial and non-
financial incentives to encourage immunisation and
other preventive health behaviours. Non-randomised
studies of the measles campaign in Africa suggest that
coupling the distribution of vaccines and bed nets
increases ownership of bed nets by more than 40 per-
centage points,
78
but the studies did not estimate the
programmes impact on measles vaccination rates. In
Nicaragua, attendance at an immunisation campaign
increased from 77% to 94% when food incentives
equivalent to about three to five days of wages were
introduced to encourage vaccination in 1985.
9
That
study, however, was an observational study in which
the treatments were sequential rather than contem-
poraneous. Conditional cash transfer programmes,
1
Department of Economics,
Massachusetts Institute of
Technology, 50 Memorial Drive,
E52-391, Cambridge, MA 02142,
USA
2
Abdul Latif Jameel Poverty Action
Lab, Massachusetts Institute of
Technology, Cambridge, MA
3
630 West 168th Street,
Columbia University Physici ans
and Surgeons Mailbox #67, New
York, NY 10032, USA
Correspondence to: E Duflo
eduflo@mit.edu
Cite this as:
BMJ
2010;340:c2220
doi:10.1136/bmj.c2220
BMJ | ONLINE FIRST | bmj.com page 1 of 9
on 15 October 2019 by guest. Protected by copyright.http://www.bmj.com/BMJ: first published as 10.1136/bmj.c2220 on 17 May 2010. Downloaded from
popular in Latin American countries, have been effec-
tive in promoting the use of certain preventive health-
care services and have also had a positive impact on
health outcomes for women and children.
10-13
These
programmes, however, did not have a large impact
on immunisation rates.
14
The lack of impact might be
because of high initial immunisation rates in the areas
where the programmes were carried out. Most condi-
tional cash transfer programmes have been implemen-
ted in countries with existing adequate local health
infrastructure so evaluations of their impact cannot
shed light on the relative cost effectiveness of establish-
ing incentive based interventions (versus improving
quality of health infrastructure only) in more resource
poor settings.
10
We assessed the relative efficacy and cost effectiveness
of improving the supply of infrastructure for immunisa-
tion only compared with improving supply and simulta-
neously increasing demand through the use of incentives.
The study was a partnership between the Abdul Latif
Jameel Poverty Action Lab (J-Pal) at the Massachusetts
Institute of Technology and Seva Mandir, a non-govern-
mental organisation in Udaipur district.
METHODS
We used a clustered randomised controlled trial to
evaluate two interventions in rural Rajasthan, India.
In one intervention (A), regular, well publicised immu-
nisation clinics (referred to as camps) were held,
while in the second intervention (B), similar camps
were held and parents were also offered small incen-
tives to immunise their children. A third set of villages
formed the control group. We used a cluster level
design because individual level randomisation could
have generated resentment against the non-govern-
mental organisation.
Selection and description of sample
The main sample comprised 134 villages randomly
selected from a Seva Mandir catchment area in Udai-
pur by using a computer generated random variable
and over-sampling villages far away from a road. We
also randomly selected one village within 6 km of each
intervention village to measure potential spillover of
the interventions.
Within each village, a household census was con-
ducted, and 30 households containing children aged
0-5 years were randomly selected with a random num-
ber generator to be part of the sample. The same house-
holds were surveyed again at the end point. The
criterion for inclusion of a child in this study was to
belong to a sampled household and to be aged 1-3 at
the end point of the study (main sample) or to have
been aged 0-6 months at baseline (baseline cohort).
As Seva Mandir works in poorer and more tribal
villages, the villages selected are not representative of
Udaipur in general but are representative of Seva Man-
dirs catchment area, an impoverished area where
health status is poor.
15
The public facilities serving
these areas are characterised by high absenteeism:
weekly visits during the year preceding the
intervention showed that 45% of the health staff in
charge of immunisations (auxiliary nurse midwives)
were absent from their village level health centre (and
could not be found anywhere in the village) on any
given workday, and there was no predictable pattern
to their absence.
11
Interventions
In this study children received the WHO/Unicef
extended package of immunisation, provided by the
Indian government. This includes one dose of BCG
vaccine, three doses of DPT (diphtheria-pertussis, teta-
nus) vaccine, three doses of oral polio vaccine, and one
dose of measles vaccine.
16
A child should be fully
immunised (that is, have received all the vaccines in
the extended package) by the age of 1 year.
Given that a full immunisation course requires at
least five visits to a public health facility, the unreliabil-
ity of the auxiliary nurse midwives might deter families
from taking their children to the centre to complete the
full immunisation schedule. Therefore, intervention A
(immunisation camps) focused on establishing regu-
lar availability of immunisation services. It consisted of
a mobile immunisation team, including a nurse and
assistant (both hired by Seva Mandir), who conducted
monthly immunisation camps in the villages. The
nurse and assistant held the camp on a fixed date
every month at a fixed time (11 am to 2 pm). The pre-
sence of the nurse and assistant was verified by the
requirement of timed and dated pictures of them in
the villages and by regular monitoring. Review of
records showed that of 1336 planned camps, 95%
(1269) took place. In addition, in each village a social
worker was responsible for identifying children,
informing mothers about the availability of the immu-
nisation camps, and educating them about the benefits
of immunisation. Seva Mandir has been active in the
area for over 50 years and is trusted in villages. While
this programme was new, and Seva Mandir had not
been engaged in any effort to provide immunisation
before, the health unit of the organisation had been
conducting several programmes in these villages or in
neighbouring villages, most notably an effective pro-
gramme of training traditional birth attendants for
delivery. Many villages also already had health social
workers, responsible for general health advice, infor-
mation on prenatal and other preventive care, referral
to health centres, and help with the DOT (directly
observed therapy) programme. The organisation thus
benefits from a high level of trust among villagers,
which might have ameliorated issues of mistrust that
surround immunisation programmes in India.
17
Intervention B used the same infrastructure as inter-
vention A but in addition offered parents 1 kg of raw
lentilsperimmunisation administeredanda set of thalis
(metal plates used for meals) on completion of a childs
full immunisation. The value of the lentils was about 40
rupees (about $1), equivalent to three quarters of one
days wage, and the value of the thalis was about 75
rupees. Seva Mandir decided to provide lentils, rather
than cash, as this was useful to the family and also
RESEARCH
page 2 of 9 BMJ | ONLINE FIRST | bmj.com
on 15 October 2019 by guest. Protected by copyright.http://www.bmj.com/BMJ: first published as 10.1136/bmj.c2220 on 17 May 2010. Downloaded from
immediate had nutritional value. The amount roughly
corresponds to the opportunity cost of time for the
mother. The thalis were chosen as a tangible sign of
achievement, while also being of immediate use.
At the first immunisation, every child was given an
official immunisation card indicating their name, the
name of their parent/s, and the date and type of each
immunisation performed. The nurse also kept a
detailed logbook. Following standard guidelines,
when a child arrived at a camp without an immunisa-
tion card and it could not be ascertained whether he or
she had received a given immunisation, the child was
immunised.
18
Study and evaluation design
We evaluated the impact of the interventions using a
clustered randomised control trial. Using the random
number generator in the statistical package Stata (ver-
sion 9), and after stratification by geographical block
(the administrative unit above the village), one author
(ED) randomly selected 30 of the 134 study villages to
receive intervention A and 30 to receive intervention
B. The 74 remaining villages were control villages and
received no additional intervention. Naturally, given
the context and the intervention, the allocation of vil-
lages to treatment or control was not blind. In most
control villages, a Seva Mandir health worker was pre-
sent and encouraging uptake of preventive services,
including immunisation, was part of their mission. Vil-
lagers were to obtain immunisation in the closest
health centre. In all villages, the government nurse
continued to provide immunisation services for the
duration of the study. When a camp was established
in a village, any non-immunised child younger than
5, from any village, was eligible for immunisation in
the camp. All children younger than 2 were eligible
for the lentils in intervention B camps, irrespective of
the village they came from. (Villages from all three
treatment groups were sufficiently far from each
other (over 20 km) so we expected no contamination
between the villages.) Children who began the immu-
nisation course before turning 2 remained eligible for
the incentives until the completion of the immunisa-
tion course. Therefore children included in the study
sample were aged 1-3 at the end point survey.
Data sources
Our primary outcome was the receipt of all or part of
the extended package of immunisation. Most house-
holds in the areas did not have immunisation cards so
we ascertained the immunisation rate during inter-
views with the mother. Mothers were surveyed about
the immunisation status of all children aged under 7 at
the end point and about her immunisation status dur-
ing her pregnancy with each child.
We developed our survey instrument to elicit accu-
rate reporting of immunisation status, regardless of
where the immunisation was obtained (camp, health
centres, private doctor). Because a parent might con-
fuse immunisation with other injections (injection of
antibiotics is common in India) and might not realise
the difference between different vaccinations, the
instrument asked in detail about each injection
received by the child, including whether it was admi-
nistered to cure a sickness, in which facility it was admi-
nistered, and where on the body it was injected. In
addition, at the end of the survey, the field officer
examined the child to ascertain whether they had the
distinctive scar left by the BCG vaccine. If the family
had a vaccination card for the child researchers also
recorded the information on the card.
Self report can be influenced by recall bias (mothers,
who are often illiterate, might not remember) and
potentially by social desirability bias, which might be
affected by their intervention group. We carried out
several validation exercises in which we compared
the self reports with the BCG scar, the immunisation
card (available for 343 children), and a sample of chil-
dren from intervention villages. The immunisation
camp records were matched with the survey data.
BCG self report seemed to be accurate (see appendix
1 on bmj.com). Immunisation status elicited from the
survey instrument corresponded to within one injec-
tion of the status indicated on the card 79% of the
time and to within one injection of the status indicated
in the logbook 74% of the time. The mis-measurement
was not correlated with the treatment statusor the num-
ber of immunisations reported and was not systemati-
cally over-reported or under-reported and should
therefore increase noise but not necessarily introduce
bias (as measurement error is the dependent variable
and is not differential in different treatment groups).
While self reports of immunisation from mothers
who do not have an immunisation card is not perfect,
a meta-analysis of several validation studies has shown
that they are generally of acceptable quality and that
they represent the best available independent mea-
sure of DPT3 coverage.
4
Nevertheless, to verify that theresults were not biased
by the use of self reported data, we carried out two addi-
tional analyses: an analysis of the presence of BCG scar
(observed by the field officer at the end of the mothers
interview) and an analysis based on the administrative
data maintained in intervention A and intervention B
villages. For these children, a complete computerised
record of immunisation received in the camp and else-
where is maintained. This allows a comparison of inter-
vention A and B villages for childrenwhohave received
at least one immunisation in the camp. For this analysis,
we also focused on children aged 1-3.
The baseline survey took place between June 2004
and February 2005 and covered all children aged 0-5
in the sample households. The intervention started in
each geographical block after the baseline investiga-
tions. The end point survey took place between July
2006 and February 2007, about 18 months after the
interventions started in a particular village. It used the
same survey instrumentsand coveredall children age 0-
7 in the same households to identify eligible children (1-
3 at end point, or 0-6 monthsat end point). Bothsurveys
were blind: interviewers did not know which villages
belonged to which intervention (or control) group.
RESEARCH
BMJ | ONLINE FIRST | bmj.com page 3 of 9
on 15 October 2019 by guest. Protected by copyright.http://www.bmj.com/BMJ: first published as 10.1136/bmj.c2220 on 17 May 2010. Downloaded from
Data entry, cleaning, and validation were completed in
September 2007 and analysis and report write up took
place between October 2007 and May 2008. The paper
was revised a first time between September 2009 and
November 2009 and a second time in March 2010.
End points
In the household sample, our main analysis focused on
children aged 1- 3 at the end point (that is, eligible and
old enough to be fully immunised). The outcomes
include the probability of receiving at least one immu-
nisation (excluding oral polio vaccine, which almost all
children have received and therefore does not affect the
statistics); the presence of the BCG scar; the number of
immunisations received; and the probability of receiv-
ing the complete extended package of immunisation.
We also analysed the administrative data collected in
the camp: the outcomes were the number of immuni-
sations received by children who received at least one
immunisation in the camp, and the probability of
receiving all the vaccines in the extended package of
immunisation.
Complementary analysis reports the impact of the
interventions on neighbouring villages. We report the
probability of being immunised in villages neighbour-
ing intervention A and intervention B camps, differ-
ences between these two groups of neighbouring
villages and the control group, and relative risks.
Costs
We carried out a cost effectiveness analysis of both
interventions (see appendix 2 on bmj.com for details).
Statistical analysis
Taking into account correlation of the end point within
a village and clustering of the treatment at that level (a
intracluster correlation of 0.25 was assumed based on a
preliminary survey) and given a baseline immunisation
rate of 2% in the control group, we determined that a
sample of 30 villages per treatment arm, with a random
sample of 30 households per village (assuming about
1.4 children aged 1-3 years surveyed in each house-
hold), was sufficient to obtain 80% power for a 5%
level testof a differenceof at least five percentage points
in the probability of being fully immunised between
any two groups (treatmentA, treatmentB, and compar-
ison). The larger control group increases power.
We used intention to treat analysis
that is, all house-
holds were analysed with the assumption that they
remained in the treatment group to which they were
initially assigned. For the binary variables, we report
proportion in each group, difference in proportions
across groups, and relative risks. For the count vari-
ables (number of immunisation), we report values in
the treatment group, difference across groups, and rela-
tive risks. The analysis adjusts for clustering at the vil-
lage and the family level. For the difference in
proportion, we used a multilevel mixed effect linear
model of the probability of being immunised on the
treatment indicator, with a hierarchical error structure
that allows cluster level heterogeneity (random effect)
at the village and at the family level. For the relative
risk, we used a multilevel mixed effect Poisson model
with the same hierarchical error structure. We did not
include any control variables. In all the analyses we
used Stata version 10.
Assessed for eligibility (n=134 villages, each with 30 households with children aged 0-7 years at end point)
Randomised
Allocated to intervention A (n=453
households in 30 villages)
All received allocated intervention
Lost from baseline to end point (n=71
households)
Allocated to intervention B (n=481
households in 30 villages)
All received allocated intervention
Lost from baseline to end point (n=82
households)
Allocated to control (n=1224 households
in 74 villages)
All received control
Lost from baseline to end point (n=210
households)
Enrolment
Allocation
Followed up to end point (n=379 children
aged 1-3 in 30 villages)
Estimated loss to follow-up (n=22 children;
household moved, unable to find child,
child died)
Discontinued intervention (n=0)
Followed up to end point (n=382 children
aged 1-3 in 29 villages)
Estimated loss to follow-up (n=13 children;
household moved, unable to find child,
child died)
Discontinued intervention (n=0)
Followed up to end point (n=866 children
aged 1-3 in 69 villages)
Lost to lack of covariates (n=6 children)
Estimated loss to follow-up (n=44 children;
household moved, unable to find child,
child died)
Discontinued intervention (n=0)
Follow-up
Analysed (n=379 children, none excluded) Analysed (n=382 children, none excluded) Analysed (n=860 children, none excluded)
Analysis
Excluded (n=4246):
Did not meet inclusion criteria (too old or too young at end point) (n=4246)
Refused to participate (n= 0)
Other reasons (n=0)
(randomisation occurred at village level; all villages assigned to receive treatment did so)
Fig 1
|
Flow of participants through study
RESEARCH
page 4 of 9 BMJ | ONLINE FIRST | bmj.com
on 15 October 2019 by guest. Protected by copyright.http://www.bmj.com/BMJ: first published as 10.1136/bmj.c2220 on 17 May 2010. Downloaded from
RESULTS
The final sample in the household surveys included
2188 children aged 1-3 years at the end point from
2898 households in five groups of villages: 74 control
villages (n=860 children), 30 villages in intervention
group A (n=379), 30 in intervention group B (n=382),
27 neighbouring an intervention group A village
(n=265), and 26 neighbouring an intervention group
B village (n=302) (fig 1).
Baseline characteristics of children across the treat-
ment arms were comparable (table 1). There were no
differences in proportions of children partially or fully
immunised at baseline or in the covariates. Immunisa-
tion rates were less than 2% among 1-3 year olds. The
intracluster correlation at baseline was 0.28.
The final sample with logbook data included 2106
children aged 1-3 who received at least one immunisa-
tion in the camps: 407 in 30 villages in intervention A
and 725 in 30 villages in intervention B.
Primary end point: impact on immunisation in treatment
village
The highest rate of full immunisation was observed for
intervention B (table 2, fig 2). In intervention B vil-
lages, 148/382 (mean 39%, 95% confidence interval
30% to 47%) children were completely immunised
compared with 68/379 (18%, 11% to 23%) in inter-
vention A villages and 50/860 (6%, 3% to 9%) in con-
trol villages (fig 2). The relative risk of being
completely immunised was 3.1 (2.0 to 4.2) for inter-
vention A versus control, 6.7 (4.5 to 8.8) for inter-
vention B versus control, and 2.2 for intervention B
versus intervention A (1.5 to 2.8). Compared with the
control group, immunisation rates more than doubled
in intervention A villages and increased by more than
six times in the intervention B villages. There were no
adverse events (severe reaction to immunisation)
reported in either intervention groups.
The difference between intervention B and inter-
vention A was more marked for full immunisation
than for the number of immunisations received and dis-
appeared for the probability of receiving at least one
injection. Specifically, 78% (70% to 85%) of children in
intervention A villages received at least one injection
compared with 74% (67% to 82%) of children in inter-
vention B villages. Similarly, 50% (41% to 59%) of chil-
dren in intervention A villages and 50% (41%to 59%) of
children in intervention B villages had a BCG scar (v
28% (21% to 36%) in control villages) (fig 3), showing
that the impact of the incentivewasmainly to reduce the
number of children dropping out after three injections.
Over half (52%, 43% to 62%) of children who were
reported as receiving at least one injection in inter-
vention B villages were completely immunised com-
pared with 23% (15% to 32%) in intervention A.
The analysis of the logbooks confirms these results.
Of the children who received at least one injection in
the camps, 485/700 (67%, 59% to 74%) were fully
immunised in intervention B and 197/407 (48%, 38%
to 59%) were fully immunised in intervention A. The
relative risk of being completely immunised in inter-
vention B compared with intervention A for children
who received at least one injection was 1.4 (1.1 to 1.7).
The propensity for children to drop out before com-
pleting the full course, particularly in intervention A
villages, was not because the camps became less popu-
lar over time: children who received their first immu-
nisation later received on average the same number of
immunisations as children who received their first
immunisation earlier.
Impact on neighbouring villages
Table 3 shows immunisation rates in villages neigh-
bouring the intervention villages. In villages within a
few kilometres of an intervention B camp, 61/302 chil-
dren (20%, 10% to 31%) were completely immunised
compared with 36/265 children (11%, 5% to 16%) in
villages bordering an intervention A camp. These con-
fidence intervals overlap, but the relative risk is signifi-
cantly greater than one. The relative risk of being
completely immunised for villages neighbouring inter-
Table 1
|
Baseline characteristics by allocated group*. Figures are percentages of children
unless stated otherwise
Control group Treatment A Treatment B
Mean age (months) 10.2 (9.2 to 11.3) 10.4 (8.8 to 12.0) 11.06 (9.72 to
12.40)
Mean household size (No of people) 6.7 (6.5 to 7.0) 6.7 (6.3 to 7.1) 6.74 (6.46 to 7.03)
Male 0.5 (0.5 to 0.6) 0.5 (0.4 to 0.6) 0.50 (0.44 to 0.57)
Member of scheduled castes/scheduled
tribes (disadvantaged group)
0.9 (0.8 to 1.0) 0.9 (0.8 to 1.0) 0.96 (0.8 to 1.02)
Literate head of household 0.4 (0.3 to 0.4) 0.4 (0.3 to 0.5) 0.37 (0.27 to 0.47)
Monthly household income
2858.70 (2433.17
to 3284.23)
3196.57 (2743.95
to 3649.18)
2729.09 (2374.79
to 3083.39)
Land size owned by family (in bighas
) 3.9 (3.5 to 4.3) 4.0 (3.5 to 4.5) 3.51 (2.98 to 4.04)
Below poverty line 0.5 (0.5 to 0.6) 0.5 (0.4 to 0.6) 0.50 (0.42 to 0.59)
No of rooms in house 2.0 (1.8 to 2.2) 2.1 (1.8 to 2.4) 1.90 (1.74 to 2.06)
House has electricity 0.2 (0.1 to 0.2) 0.2 (0.1 to 0.3) 0.06 (0 to 0.12)
Treats water 1.1 (1.1 to 1.2) 1.1 (1.0 to 1.1) 1.08 (1.03 to 1.12)
No of immunisations 0.6 (0.5 to 0.8) 0.8 (0.5 to 1.1) 0.45 (0.25 to 0.65)
Completely immunised 0.01 (0 to 0.01) 0.02 (0 to 0.04) 0.00 (0 to 0.02)
At least one injection 0.3 (0.3 to 0.4) 0.4 (0.3 to 0.5) 0.30 (0.19 to 0.4)
*A
=
reliable immunisation camps, 30 villages; B
=
reliable immunisation camps with incentives, 30 villages;
control
=
no treatment, 74 villages.
In rupees (1000 rupees
=
about
£
15,
17, $23).
Generally <1 acre (0.4 hectare).
Fully immunised (%)
Control
villages
0
10
20
30
40
50
Intervention
A villages
(reliable
camps)
Intervention
B villages
(reliable
camps)
Villages
neighbouring
intervention
A camps
Villages
neighbouring
intervention
B camps
Fig 2
|
Percentage of children aged 1-3 years fully immunised
by treatment status
RESEARCH
BMJ | ONLINE FIRST | bmj.com page 5 of 9
on 15 October 2019 by guest. Protected by copyright.http://www.bmj.com/BMJ: first published as 10.1136/bmj.c2220 on 17 May 2010. Downloaded from
vention B camps versus those neighbouring inter-
vention A camps was 1.9 (1.1 to 2.8) (table 3).
This analysis is confirmed in the logbook data,
where results are more precise. In villages within a
few kilometres of an intervention B camp, 452/700
children who received at least one immunisation
(65%, 58% to 72%) were completely immunised com-
pared with 69/208 children (33%, 23% to 44%) in vil-
lages bordering an intervention A camp.
Costs
The average cost to Seva Mandir of fully immunising a
child was $27.94 (1102 rupees, about £16 or 19) in the
reliable camp with incentives and $55.83 (2202 rupees)
in the reliable camp without incentives (see appendix 2
on bmj.com for further details). The difference comes
from the fact that camps had to be open from 11 am to
2 pm, regardless of the number of children present.
Thus, the higher average number of children in the
camps with incentives spread the daily fixed cost
(mainly, the salary of the nurse and assistant) over
more children. The marginal cost of an extra child
being fully immunised with and without incentive in a
given camp is, respectively, $6.60 and $1.30. The sal-
aries of the nurse and assistant represent about half of
the cost of the camp without incentives. Monitoring
that the camps are regularly held is another important
part of the cost, which is necessary in light of high
absenteeism.
14
When we calculated the cost that would
be incurred by a government able to use an existing
health infrastructure, the average cost of fully immunis-
ing a child was estimated at $25.18 in camps without
incentives and $17.35 in camps with incentives.
DISCUSSION
Summary of results
This randomised controlled study of immunisation
camps shows that offering modest incentives to
families in resource poor settings can significantly
increase uptake of immunisation services, when reli-
able services are available. In our study, reliable
camps with incentives achieved significantly higher
rates of full immunisation for children aged 1-3 com-
pared with control areas where no camps were made
available and compared with reliable camps without
incentives. While control villages had a full immunisa-
tion rate of nearly 6%, villages in which a reliable camp
was held showed rates of around 18%, and adding the
incentive pushed rates to nearly 40%, a significant
increase. The rate achieved with incentives represents
a more than sixfold increase over the rate in control
villages, in a particularly poor and hard to reach popu-
lation. It is, however, still too low to achieve herd
immunity.
Surprisingly,despite that fact that the family received
a set of plates for the last immunisation, the biggest
increasebetweeninterventionA and interventionB vil-
lages was for the third and fourth immunisation. This
might indicatethat parents are more sensitivetothe fact
that there is an incentive than to the level of the incen-
tive, a finding consistent with the results that in condi-
Percentage
No of immunisations
0
0
20
40
60
80
1 2 3 4 5
Intervention A main villages
Intervention B main villages
Fig 3
|
Number of immunisations received by children aged
1-3 years
Table 2
|
Effects of group allocation (A
=
immunisation, B
=
immunisation plus incentive). Numbers are in absolute values
Mean (95% CI) Difference* (95% CI) Relative risk
(95% CI)
Control A B A
control B
control B
AAv control B v control B v A
End point cohort (aged 1-3 years)
No in group 860 379 382
——— ——
No of immunisations 1.20
(0.94 to 1.46)
2.35
(1.99 to 2.71)
2.85
(2.44 to 3.25)
1.15
(0.95 to 1.35)
1.70
(1.48 to 1.92)
0.55
(0.26 to 0.83)
2.14
(1.84 to 2.44)
2.66
(2.28 to 3.05)
1.22
(1.08 to 1.36)
1immunisation 0.49
(0.40 to 0.57)
0.78
(0.70 to 0.85)
0.74
(0.67 to 0.82)
0.29
(0.23 to 0.35)
0.26
(0.20 to 0.33)
0.03
(
0.09to0.04)
1.59
(1.35 to 1.83)
1.52
(1.29 to 1.75)
0.96
(0.80 to 1.11)
Has BCG scar
0.28
(0.21 to 0.36)
0.50
(0.41 to 0.59)
0.50
(0.41 to 0.59)
0.22
(0.15 to 0.28)
0.22
(0.15 to 0.28)
0.00
(
0.08to0.08)
1.76
(1.41 to 2.12)
1.76
(1.41 to 2.12)
1.00
(0.79 to 1.21)
Completely immunised 0.06
(0.03 to 0.09)
0.18
(0.11 to 0.25)
0.39
(0.30 to 0.47)
0.13
(0.09 to 0.16)
0.34
(0.30 to 0.38)
0.21
(0.15 to 0.28)
3.09
(1.96 to 4.21)
6.66
(4.53 to 8.80)
2.16
(1.54 to 2.78)
Logbook cohort (aged 1-3 years)
No in group
407 725
——— ——
Total No of immunisations
3.70
(3.39 to 4.01)
4.18
(3.99 to 4.37)
——
0.59
(0.25 to 0.93)
——
1.15
(1.05 to 1.25)
Completely immunised
0.48
(0.38 to 0.59)
0.67
(0.59 to 0.74)
——
0.22
(0.10 to 0.33)
——
1.43
(1.12 to 1.73)
*Estimated by fitting multilevel mixed effect linear model, with clustering at hamlet and household level.
Estimated by fitting multilevel mixed effect Poisson regression, with clustering at hamlet and household level.
For analysis with BCG scar as outcome, there were 790 observations in control group, 334 in treatment A group, and 336 in treatment B group.
RESEARCH
page 6 of 9 BMJ | ONLINE FIRST | bmj.com
on 15 October 2019 by guest. Protected by copyright.http://www.bmj.com/BMJ: first published as 10.1136/bmj.c2220 on 17 May 2010. Downloaded from
tional cash transfer programme, the size of the transfer
does not seem to matter beyond the fact that there is a
positive transfer,
19 20
and suggests that larger incentives
mightnotincrease the immunisationratemuchbeyond
what was found here. Moreover, while a camp without
incentives increased immunisation rates only in the vil-
lage where it took place, camps with incentives also
increased rates in neighbouring villages.
Comparison with other studies
Several previous studies have shown that uptake of pre-
ventive behaviours is sensitive to small incentives or
small costs, suggesting that incentives can play a role in
promoting preventive health services.
9192122
Other
researchers have suggested that in resource poor set-
tings, ensuring a reliable supply of health services and
educating parents about the benefits of preventive care
are more important than providing incentives.
923
Our
findings contrast with those of a study in Timor Leste,
which found that food incentives did not increase the
completion of tuberculosis treatment.
24
This difference
might be attributable to various factors: the frequency of
the treatment (vaccines versus daily treatment); the fact
thatadults weretargetedinthetuberculosisintervention;
the relatively high initial compliance rate; and the differ-
ence in demand for treatment and prevention. More stu-
dies are needed to understand in what conditions
incentives for parents or patients have a positive impact.
Limitations of study
We relied in part on self reported data on immunisa-
tion, though the results were robust when we used the
BCG scar or the immunisation reported in the logbook
as outcomes. The study was not blind from the point of
view of the villagers, who might have been motivated
to attend the camp to ensure that Seva Mandir would
not cancel the programme. The study was also con-
ducted in an environment with low density where
initial immunisation rates were extremely low. In
areas where initial immunisation rates are higher, simi-
lar interventions might not produce as dramatic an
increase. On the other hand, the impacts might be
much higher and the costs much lower in more densely
populated regions. Finally, we could look only at the
joint effect of improving supply and incentivising
demand. Because it would have been impossible to
administer in practice, we could not introduce an
incentive programme without simultaneously making
the supply more reliable.
Policy implications
The magnitude of our results shows that, in this setting,
holding regular immunisation camps combined with
incentives improves immunisation rates, even though
the rate of full immunisation remained low. Moreover,
providing incentives and improving the supply of ser-
vices was also more cost effective than improving the
supply of services alone. The cost of immunising with
incentive (around $17.35) is higher than the cost cur-
rently in the Indian budget (about $4 per immunisa-
tion, according to the healthcare budget
25
), but it is of
an order of magnitude comparable with the payment
from the Global Alliance for Vaccines and Immunisa-
tion (GAVI) to member countries of $20 per extra
child who would not have been immunised otherwise.
This is a relevant comparison as these kinds of pro-
gramme would target difficult to reach children
who are not already immunised under Indias standard
programme. A review of interventions to improve
immunisation found that the cost of most interventions
is about $10-20 per newly immunised child.
26
More-
over, while the lentils represented a cost to Seva Man-
dir, their distribution could have led to improved
nutrition for the family in an environment where mal-
nutrition and anaemia are endemic, and it is not clear
they should be considered as a cost.
Table 3
|
Effects of treatment in villages adjacent to intervention or control villages (A
=
immunisation, B
=
immunisation plus incentive)
Mean in group (95% CI) Difference* (95% CI) Relative risk
(95% CI)
Control Treat A B A
control B
control B
AAv control B v control B v A
End point cohort (aged 1-3 years)
No in group 860 265 302
——— ——
No of immunisations 1.20
(0.94 to 1.46)
1.41
(0.94 to 1.88)
1.70
(1.18 to 2.22)
0.23
(0.00 to 0.46)
0.58
(0.35 to 0.82)
0.36
(0.02 to 0.69)
1.18
(0.92 to 1.43)
1.48
(1.18 to 1.77)
1.26
(0.93 to 1.60)
1immunisation 0.49
(0.40 to 0.57)
0.49
(0.34 to 0.64)
0.51
(0.39 to 0.63)
0.01
(
0.06to0.09)
0.05
(
0.02to0.12)
0.03
(
0.06to0.12)
1.00
(0.80 to 1.19)
1.05
(0.86 to 1.24)
1.05
(0.81 to 1.30)
Has BCG scar
0.28
(0.21 to 0.36)
0.28
(0.16 to 0.41)
0.30
(0.18 to 0.41)
0.01
(
0.06to0.08)
0.03
(
0.04to0.10)
0.02
(
0.07to0.10)
1.00
(0.73 to 1.28)
1.05
(0.78 to 1.32)
1.05
(0.70 to 1.39)
Completely immunised 0.06
(0.03 to 0.09)
0.11
(0.05 to 0.16)
0.20
(0.10 to 0.31)
0.05
(0.01 to 0.09)
0.16
(0.12 to 0.20)
0.11
(0.05 to 0.18)
1.83
(0.93 to 2.73)
3.47
(2.18 to 4.77)
1.91
(1.06 to 2.77)
Logbook cohort (aged 1-3 years)
No in group
208 700
——— ——
Total No of immunisations
3.23
(2.91 to 3.55)
4.15
(3.95 to 4.35)
——
0.92
(0.50 to 1.35)
——
1.28
(1.14 to 1.42)
Completely immunised
0.33
(0.23 to 0.44)
0.65
(0.58 to 0.72)
——
0.30
(0.15 to 0.45)
——
1.91
(1.36 to 2.45)
*Estimated by fitting multilevel mixed effect linear model, with clustering at village and household level.
Estimated by fitting multilevel mixed effect Poisson regression, with clustering at village and household level.
For the analysis with BCG scar as outcome, there were 790 observations in control group, 239 in treatment A group, and 252 in treatment B group.
RESEARCH
BMJ | ONLINE FIRST | bmj.com page 7 of 9
on 15 October 2019 by guest. Protected by copyright.http://www.bmj.com/BMJ: first published as 10.1136/bmj.c2220 on 17 May 2010. Downloaded from
One implication of the increased immunisation rates
in the villages with camps with incentives (and in the
surrounding villages) is that they were busier than
those without incentives. Inspection of the logbook
showed that for any given camp, each day an average
of 4.5 immunisations were given without incentives
and 13.4 with incentives.
Interpretation, unanswered questions, and future research
Our results also suggest reasons that immunisation has
not been more widely embraced in developing coun-
tries. Previous work has emphasised the need to
strengthen health systems to achieve the millennium
development goals.
27
Our results suggest that to
achieve this strengthening, improving the supply
alone might not be enough: even a fully reliable supply
system has a relatively modest effect on uptake of
immunisation. In intervention A, even when access
was good and a social worker constantly reminded
parents of the benefits of immunisation, more than
80% did not get their children fully immunised. Never-
theless, more than 75% obtained the first injection
without the incentive and stopped attending the
camps only after two or three injections. This shows
that the parents do not have strong objections or fears
about immunisation, but that they are not persuaded
enough about its benefits to overcome the natural ten-
dency to delay a slightly costly activity (immunisation
is free, but it takes some time and effort to go to the
centre and get the child immunised, and the child
might have a fever afterwards). This fits the findings
of sociological research in India, where nurses describe
parents forgetting to bring their children to the immu-
nisation day, and where they are particularly careful to
manage even benign side effects of immunisation to
avoid discouraging parents from coming back.
5
It
also explains the tendency for children not to complete
the whole course of immunisation. Providing the len-
tils helps to overcome this procrastination because the
lentils make the occasion a small plus rather than a
small minus. Thus, in the case of preventive care,
small barriers might turn out to have large implica-
tions. Finding effective ways to overcome small bar-
riers might hold the key to large improvements in
immunisation rates and uptake of other preventive
health behaviours. In the case of immunisation, small
incentives coupled with regular delivery of services
seem to have the potential to play this role.
While we primarily examined the effect of small
incentives and supply improvement when they are cor-
rectly implemented, we need to know whether and
how such an incentive programme could be general-
ised. Under the National Rural Health Mission, the
government of India now has a health worker in each
village who is responsible for encouraging uptake of
preventive care. Furthermore, several Indian states,
including those with comparatively low immunisation
rates (Orissa, Bihar, Rajasthan), are already imple-
menting a camp approach, where the regular auxili-
ary nurse midwife immunises children in villages on a
rotating schedule. We are hoping to conduct an impact
evaluation of the addition of small incentives to parents
in this structure, in collaboration with the state govern-
ment in India, to evaluate the potential of these types of
intervention to be adopted as large scale policies and
the challenges that would need to be overcome.
We tha nk Je nnifer Tobin for her help in editing this manus cript for
publication. She was funded by the Abdul Lati f Jameel Poverty Action lab.
Contributors: AVB, ED, and RG participa ted in the study design. ED a nd DK
completed the data analysis. All authors participated in data collection,
data interpret ation, a nd drafting of the manuscript. E D is guarantor.
Funding: T his study was funded by the Mac Arthur Fo undation. All
researchers declare that the research was entirely independent f rom the
funders. The funders had no involvement in t he design and conduct of the
study; collection, management, analysis, and interpretation of the data;
and preparation , review, or approval of the manuscript. The intervention
was funde d by the Evangelischer E ntwicklungdi enst (Germany), I nter
Church Cooperation for Development Cooperation (Netherlands), and
Plan International, through Seva Mandir comprehensive plan. None of the
funding organ isations participated in the design of the study (although
the MacAr thur Foun dation reviewed the design before making the
funding de cision), the data collection or analysis, or the decision to su bmit
the paper for publication.
Competi ng interests: All authors have completed the Unified Competing
Interest form at www.icmje.org/coi_disclosure.pdf (available on request
from the corresponding author) and declare that they have no competing
interests relevant to this work.
Ethical approval: This study was approved by the health ministry of the
government of Rajasthan, the office on the use of human su bjects at
Massachusetts Ins titute of Technology, and the ethics committee of
Vidhya Bhawan, the university which hosted the project in Udaipur.
Informed consent was first obt ained orally at the community level fro m
the research villages through village meeti ngs to which all adult members
of the village were invited. Individua l level informed consent was then
obtained orally from every family participating in the study.
Data sharing: Statistical code an d full dataset available from the
corresponding author at eduflo@mit.edu. Consen t was not obtained, but
the presented data are a nonymised a nd risk of identification is ext remely
low.
1 WHO and UNICEF. Global immunization vision and strategy. World
Health Organization, 2005. www.who.int/vaccines-documents/
DocsPDF05/GIVS_Final_EN.pdf.
2 Bloom D, Canning D, Weston M. The value of vaccination. World
Economics 2005;6:15-39.
3 WHO and UNICEF. Global immunization data. World Health
Organization, 2008. www.who.int/immunization/newsroom/
Global_Immunization_Data.pdf.
4 Lim SS, Stein DB, Charrow A, Murray CJ. Tracking progress towards
universal childhood immunisation and the impact of global
initiatives: a systematic analysis of three-dose diphtheria, tetanus,
and pertussis immunisation coverage. Lancet 2008;372:2031-46.
5 Coutinho L, Bisht S, Raje G. Numerical narratives and documentary
practices: vaccines, targets and reports of immunisation programme.
Econ Polit Wkly 2000;35:656-66.
6 National Family Health Survey, 2005-2006 (NHFS-3). www.nfhsindia.
org/pdf/RJ.pdf.
WHAT IS ALREADY KNOWN ON THIS TOPIC
Financial incentives, such as in conditional cash transfer programmes, can be effective in
promoting the use of certain preventive healthcare services
In settings with reliable immunisation services and a high pre-existing immunisation rate
such programmes have little impact on immunisation
WHAT THIS STUDY ADDS
In a setting with a low immunisation rate (under 6%), improving the reliability of services
modestly improved uptake of immunisation
Small non-financial incentives, combined with improved reliability, had large positive
impacts on the uptake of immunisation and were more cost effective
RESEARCH
page 8 of 9 BMJ | ONLINE FIRST | bmj.com
on 15 October 2019 by guest. Protected by copyright.http://www.bmj.com/BMJ: first published as 10.1136/bmj.c2220 on 17 May 2010. Downloaded from
7 Grabowski M, Farrell N, Hawley W, Chimumbwa J, Nobiya T, Ahun M,
et al. Ghana and Zambia: achieving equity in the distribution of
insecticide-treated bednets through links with measles vaccination
campaigns. Health, nutrition, and population (HNP) discussion
paper. World Bank, 2005.
8 Wiysonge CS, Nomo E, Mawo JNN, Ticha JM. Accelerated measles
control in sub-Saharan Africa. Lancet 2006;367:394-5.
9 Loevinsohn BP, Loevinsohn ME. Well child clinics and mass
vaccination campaigns: an evaluation of strategies for improving the
coverage of primary health care in a developing country. Am J Public
Health. 1987;77:1407-11.
10 RiveraJA,Sotres-AlvarezD,HabichtJP,ShamahT,VillalpandoS.
Impact of the Mexican program for education, health, and nutrition
(Progresa) on rates of growth and anemia in infants and young
children: a randomized effectiveness study. JAMA
2004;291:2563-70.
11 Lagarde M, Haines A, Palmer N. Conditional cash transfers for
improving uptake of health interventions in low- and middle-income
countries: a systematic review. JAMA 2007;298:1900-10.
12 Fernald LCH, Gertler PJ, Neufeld LM. Role of cash in conditional cash
transfer programmes for child health, growth, and development: an
analysis of Mexico
s Oportunidades. Lancet 2008;371:828-37.
13 Glassman A, Todd J, Gaarder M. Latin America: cash transfers to
support better household decisions. In: Eichler R, Levine R, eds.
Performance- based incentives working group. performance
incentives for global health: potential and pitfalls. Brookings
Institution Press, 2009:89-121.
14 Maluccio J, Flores R. Impact evaluation of a conditional cash transfer
program: the Nicaraguan Red de Proteccion Social. Food
Consumption and Nutrition Division (FCND) Discussion Paper No
184. International Food Policy Research Institute, 2004.
15 Banerjee A, Deaton A, Duflo E. Health care delivery in rural Rajasthan.
Econ Polit Wkly 2004;39:944-9.
16 WHO. Selected vaccine introduction status into routine
immunization. World Health Organization, 2003. www.who.int/
immunization_monitoring/routine/schedule_analysis_2003.pdf.
17 Nichter M. Vaccinations in the third world: a consideration of
community demand. Soc Sci Med 1995;41:617-32.
18 Atkinson W, Pickering L, Schwartz B, Weniger B, Iskander J, Watson J.
General recommendations on immunization. MMWR Recomm Rep
2002;51:1-36.
19 Thornton R. The demand for and impact of learning HIV status:
Evidence from a field experiment. Am Econ Rev 2008;98:1829-63.
20 Filmer D, Schady N. Are there diminishing returns to transfer size in
conditional cash transfers? Policy Research Working Paper Series
4999. World Bank, 2009.
21 Kremer M, Miguel E. The illusion of sustainability. QJEcon
2007;122:1007-65.
22 Cohen J, Dupas P. Free distribution or cost-sharing? Evidence from a
randomized malaria prevention experiment. Brookings Global
Economy and Development Working Paper No 11. Brookings
Institute, 2007.
23 MorrisSS,FloresF,OlintoP,MedinaJM.Monetaryincentivesin
primary health care and effects on use and coverage of preventive
health care interventions in rural Honduras: cluster randomized trial.
Lancet 2004;364:2030-7.
24 MartinsN,MorrisP,KellyPM.Foodincentivestoimprovecompletion
of tuberculosis treatment: randomised controlled trial in Dili, Timor-
Leste. BMJ 2009;339:b4248.
25 Health Budget 2006-07: new directions? Express Healthcare
Management
2006;April. www.expresshealthcaremgmt.com/200604/
budgetanalysis01.shtml
26 Shea B, Neil A, David H. Increasing the demand for childhood
vaccination in developing countries: a systematic review. BMC Int
Health Hum Rights 2009;9(suppl I):S5.
27 Travis P, Bennet S, Haines A, Pang T, Bhutta Z, Hyder AA, et al.
Overcoming health systems constraints to achieve the Millennium
Development Goals. Lancet 2004;364:900-6.
Accepted: 11 April 2010
RESEARCH
BMJ | ONLINE FIRST | bmj.com page 9 of 9
on 15 October 2019 by guest. Protected by copyright.http://www.bmj.com/BMJ: first published as 10.1136/bmj.c2220 on 17 May 2010. Downloaded from

Discussion

Esther Duflo and Abhijit Banerjee shared the 2019 Nobel prize in Economics along with Michael Kremer. Here is a press conference at MIT with them speaking about their research. To see more about this specific paper, tune in to around minute 27: http://news.mit.edu/2019/esther-duflo-abhijit-banerjee-win-2019-nobel-prize-economics-1014 BMJ, published in the UK, is a premier medical journal, and not a typical place for economists to publish their studies. However, for this policy paper on immunization, BMJ readers are the target audience for Banerjee, Duflo, et al. These numbers have not improved significantly since this was published in 2010. 19.4 million children under the age of one year still do not receive basic vaccines, and 2-3 million people still die from diseases that can be prevent with vaccines. Source WHO: https://www.who.int/news-room/fact-sheets/detail/immunization-coverage This programs are what they sound like: the treatment is to give cash to individuals conditional on an objective (I️.e. getting immunized). Banerjee and Duflo are both part of JPAL (The Abdul Latif Jameel Poverty Action Lab). JPAL's "mission is to reduce poverty by ensuring that policy is informed by scientific evidence. They do this through research, policy outreach, and training." JPAL's website: https://www.povertyactionlab.org/ The incentives (lentils in this case) are a key treatment that they are evaluating in this case, which is why Intervention A and B are identical, save for the addition of an incentive in Intervention B. Clustered randomized controlled trials are different than individually randomized controlled trials in that rather than the randomization occurring at the individual participant level, there is a different unit of randomization. In this study, the unit of randomization occurs at the village level. This study design is popular in public health and education where it is difficult to randomize treatments to individuals. Many of the RCTs for development are longitudinal studies that take many years to complete. This study (where the baseline survey was administered in 2004 and the paper was not submitted in 2010) is not atypical for the field. The ultimate goal of JPAL and development economics is to evaluate different policies/treatments, and recommend the policies that work (based on scientific evidence from RCTs). The authors found that immunization camps with modest incentives do a great job of improving immunization rates in their trial, so they're recommending it! The program with incentives pays for itself on a per child basis. With any survey, particularly health-related surveys, there are many forms of bias that can creep into the results. In the case of recall bias: people can retrieve recollections regarding events or experiences differently (source: JPAL). Here is a list of different kinds of measurement error in surveys provided by JPAL: Completeness; Vagueness; Negatives; Overlapping Categories; Presumptions; Framing effect; Recall bias; Anchoring bias; Telescoping bias; Social desirability bias. Here is discussion of these biases in a JPAL presentation. Slides 46-70 are relevant: https://www.povertyactionlab.org/sites/default/files/documents/L2_Measurement_0.pdf This is a key visual presentation of the results. Note that the impact on neighboring village is a real example of the spillover effects of treatments. This is key. If the measurement error was correlated with the treatment outcome or dependent variable, statistical analysis and drawing conclusions from the results of the trial would be difficult. Randomized controlled trials became a popular tool for policy evaluation in the US beginning in the 1970s, and the first well known trials happened in development in the 1990s (I️.e. Kremer et al, studies on Kenya (1994)). This approach, which is the gold standard of clinical trials and was developed for evaluating medical studies, involves randomly assigning subjects into two or more groups, treating the groups differently (i.e. treatment vs. control) and comparing their outcomes. They are the gold standard because they control for many sources of bias when testing how effective treatments are. Background presentation given by Duflo about randomized controlled trials (RCTs): http://pubdocs.worldbank.org/en/394531465569503682/Esther-Duflo-PRESENTATION.pdf For more context on the invention of randomized double blind trials you should take a look at this annotated paper: [Inventing the randomized double-blind trial: the Nuremberg salt test of 1835](https://fermatslibrary.com/s/inventing-the-randomized-double-blind-trial-the-nuremberg-salt-test-of-1835)