Skip to main content
Advertisement
  • Loading metrics

Randomized Controlled Trials in Environmental Health Research: Unethical or Underutilized?

  • Ryan W. Allen ,

    allenr@sfu.ca

    Affiliation Faculty of Health Sciences, Simon Fraser University, Burnaby, British Columbia, Canada

  • Prabjit K. Barn,

    Affiliation Faculty of Health Sciences, Simon Fraser University, Burnaby, British Columbia, Canada

  • Bruce P. Lanphear

    Affiliations Faculty of Health Sciences, Simon Fraser University, Burnaby, British Columbia, Canada, Child and Family Research Institute, BC Children's Hospital, Vancouver, British Columbia, Canada

Summary Points

  • Efficacious environmental interventions are needed because environmental risks account for a large fraction of the global disease burden.
  • Randomized controlled trials have not been widely embraced by environmental health researchers and comprise less than 1% of research publications in the field.
  • Additional randomized controlled trials in environmental health would complement a strong tradition of observational research by creating new knowledge on exposure–health relationships, providing more definitive evidence of causality, identifying efficacious interventions to reduce or eliminate hazards, and countering the perception that environmental risks are evaluated with inadequate rigor.
  • Ethical issues—including clinical equipoise, the distribution of benefits and risks, and the relevance of the intervention and health outcome to the study population—must be carefully considered before conducting a randomized controlled trial of an environmental intervention.

Randomized controlled trials (RCTs) are considered the “gold standard” study design in health research. The random allocation of participants to intervention and control groups minimizes systematic differences between groups and the biases that can result. RCTs have become standard practice in the evaluation of medical and pharmaceutical treatments. In contrast, environmental (and occupational) health research has relied primarily on observational methods; randomized studies to test the effect of an environmental exposure or the efficacy of an intervention to prevent or reduce exposure are rare.

A search of PubMed articles in journals focused on medicine, environmental health, and clinical trials revealed that only 0.6% of environmental health publications since 2000 were RCTs of an intervention to reduce exposure (S1 Text). RCTs contribute a larger portion of the environmental health publications in top-ranked medical journals (4%) than in environmental health journals (0.4%)—an unsurprising result given the emphasis on clinical trials in medical research and the widespread perception that observational studies are inferior. The RCTs published to date have focused primarily on allergens, drinking water, household air pollution (HAP) from solid cooking fuels, lead, environmental tobacco smoke, and pesticides.

RCTs should be used more frequently to study environmental hazards (see Box 1). In calling for more randomized studies of interventions our objective is not to offer yet another admonishment of observational epidemiology [1][3]. The contributions that observational research has made to our understanding of environmental risks and the development of environmental health policy are impressive. Notable examples include ambient air pollution [4], lead [5], radon [6], arsenic [7], and asbestos [8], all of which are now known to cause substantial morbidity and mortality, and have policies in place to mitigate their health risks, based almost entirely on observational evidence [9]. Well-designed and carefully conducted randomized trials would complement this strong tradition of observational research. The fundamental advantages of randomized designs, such as minimization of confounding bias, are described elsewhere and need not be reiterated here. Instead, we aim to highlight how RCTs might be beneficial to environmental health research and describe some considerations for the appropriate use of RCTs to assess environmental risks and the efficacy of interventions.

Box 1. Randomized Controlled Trials

The RCT is a powerful research design that may have applicability to a wide range of risk factors in the physical, built, and social environments. RCTs may be appropriate and should be considered when the following criteria are met: (1) there is uncertainty about which (if any) intervention is most effective (i.e., clinical equipoise) or the gold standard intervention is not being implemented; (2) the intervention to be studied is feasible and affordable to the local community; (3) the intervention addresses a health risk affecting the local community; and (4) the health outcome and the timescale of the exposure–response relationship can feasibly be studied using the RCT design.

Why Are More RCTs Needed?

Interventions to reduce or eliminate environmental exposures are urgently needed; environmental risks account for 13%–37% of the disease burden (quantified by disability-adjusted life years) in individual countries [10],[11]. The individual-level health risks of environmental exposures are often modest, but the population-level impacts are substantial because exposures are highly prevalent or ubiquitous and contribute to common diseases and disabilities [9]. Environmental exposures affect health in both high-income countries and low- and middle-income countries (LMIC), although the relative importance of specific risk factors and the magnitude of the risks vary with economic development [12].

RCTs can generally provide more definitive evidence of causality than observational studies. As a result, greater use of RCTs in environmental health would help to emphasize prevention over treatment by altering the perception that environmental risks are evaluated less rigorously than medical and pharmaceutical interventions. As previously noted in the context of HAP, the perception that environmental interventions are evaluated with insufficient rigor has important implications for the allocation of limited resources:

“[Randomized] studies would go a long way in making the argument about causality to health ministries and international agencies that support them, who usually have very limited resources to deal with a number of large health problems. They have the results of rigorous studies focused on other means of dealing with these same diseases… At present, as the effectiveness and cost of such measures has been much better established, [HAP] interventions garner little attraction. On the other hand, the currently available interventions are clearly imperfect and will not serve to entirely control the diseases” [13].

Environmental interventions are cost-effective investments [14]. For example, each dollar invested to control lead hazards in the United States would result in benefits of $17–$221 [15]. By 2020 the cumulative benefits of the 1990 US Clean Air Act Amendments are projected to outweigh costs by a ratio of 30 to 1 [16]. Globally, the annual economic benefit of halving the population without access to improved cooking stoves is estimated at $105 billion compared to a net cost of $34 billion [17].

Well-designed RCTs can also identify inefficacious or even harmful interventions. For example, several studies reported that residential lead abatement increased children's blood lead levels, which led to post-abatement lead-dust standards to protect children from the short-term risks of lead released during abatement or renovation [18],[19]. Even if they do not increase health risks, non-efficacious interventions can be a waste of scarce resources and provide a false sense of safety [20],[21]. In the absence of evidence, individuals will often turn to unproven solutions—like surgical masks, which are commonplace in many cities with poor air quality but do little to reduce health risks from air pollution [22].

Individuals Versus Populations

Most environmental health RCTs conducted to date have evaluated interventions on individuals or households, but RCTs may also be appropriate for evaluating interventions implemented at the community level. For example, Patil and colleagues conducted an RCT in 80 villages to evaluate the effect of India's Total Sanitation Campaign (TSC) on the availability of improved sanitation, open defecation behaviors, water quality, and childhood diarrheal and gastrointestinal illnesses [23]. Villages randomized to the intervention group received the TSC, while control group villages received the TSC after the trial was completed. In other situations, it may be infeasible (or unethical) to randomize communities to the intervention, and “natural experiments” evaluating temporal changes in exposure and health may be a more appropriate design [24][27].

From a public health perspective, programs that reduce pollution emissions and exposure among large populations will always be preferable to interventions that attempt to reduce exposure at the individual or household level after the pollutants have been widely distributed. Nevertheless, RCT evaluations of interventions at the individual or household level can have tremendous value. Robust evidence that exposure reductions lead to improvements in health—even at the individual level—may make the argument for policy interventions at the population level more persuasive. The simplicity of the RCT design also makes it relatively easy to communicate results to non-researchers. Thus, even if the intervention under study is not feasible on a large scale, the information generated on exposure–health relationships may have relevance to population health and policy. Using RCTs to demonstrate efficacy for individual- or household-level interventions can also be valuable if emissions or production cannot be directly controlled. For example, wildfire smoke and radon are naturally occurring pollutants for which risk management involves exposure reduction primarily at the household level [28],[29].

RCTs are ideal for demonstrating efficacy (performance of the intervention under optimal conditions) but are generally inadequate for evaluating effectiveness (performance of the intervention under “real world” conditions) [30]. In clinical trials external validity is often limited because the demographics and health of study populations differ from their target populations [31]. External validity for RCTs in environmental health can be further influenced by the complexity of “scaling-up” interventions to a larger population [32]. An intervention's impact on health at the population level depends on efficacy, but also on user compliance, delivery, programming, and government policy [33]. Thus, RCTs generally represent the first step in developing an effective environmental intervention program.

Some have suggested that the relatively small study populations in RCTs are only capable of detecting “large” health effects and that RCTs are therefore not useful in public (or environmental) health research [34]. However, this is not necessarily the case because the small populations in RCTs are offset by three advantages. First, the smaller populations in RCTs often allow for refined, individual-level exposure assessment using environmental measurements and/or exposure biomarkers that is often not possible in larger observational studies. A reduction of non-differential exposure misclassification can dramatically enhance our ability to detect relationships with health outcomes. Second, the ability to detect relationships between exposure and health depends, in part, on the size of the exposure gradient in the population. Ironically, it is the ubiquity of many environmental exposures that makes their relationship with health difficult to uncover because “the hardest cause to identify is one that is universally present, for then it has no influence on the distribution of disease” [35]. RCTs can substantially increase the exposure gradient in the study population—and thus our ability to detect associations with health—by reducing exposure in the intervention group. Finally, environmental health researchers often struggle to identify the causative agent because many exposures share common sources and are therefore correlated. By reducing one specific exposure we can decrease its correlations with other exposures and more clearly identify the key agent(s) impacting health. For example, high efficiency particulate air (HEPA) filters reduce particulate matter (PM) air pollution concentrations indoors, but they have little effect on gaseous pollutants [36]. Thus, studies demonstrating health benefits from HEPA filtration provide evidence that PM plays an important role [36][38].

Ethical Considerations

RCTs raise important ethical questions, and while a comprehensive review of ethical considerations is beyond the scope of this essay, some issues with relevance to environmental health research should be mentioned. The fundamental question with all RCTs is whether it is ethical to provide a potentially beneficial treatment or intervention to some participants but not to others [39]. It is widely accepted that RCTs are only ethical in situations of clinical equipoise—genuine uncertainty among the community of experts about which (if any) intervention is most effective [40]. No participant in a randomized trial should receive a placebo or an inferior intervention if an efficacious intervention has been identified [41]. It is often assumed that participants assigned to the intervention group in an RCT will benefit, but if equipoise exists the intervention is as likely to be non-efficacious as it is to be beneficial [20],[42], and in some cases the intervention may unintentionally increase exposure [19],[43].

Research participants should receive a fair share of the benefits from the research and the benefits to participants and society should be proportional to or outweigh the risks [39],[44]. In addition, the research must address a health problem of relevance to the population under study and the population should be selected based on the research objectives, not the population's vulnerability [45]. These considerations also have applicability to RCTs conducted in LMIC. While many clinical trials are now conducted in LMIC for financial reasons [46], many randomized studies of environmental interventions focus on LMIC because that is where the public health burden of many environmental risks is greatest and where the risks are often concentrated at the household level [9],[12]. These studies are ethical only if the intervention being tested is feasible and affordable in the local context and addresses a health risk of relevance to the population under study. Finally, researchers should clearly communicate that the study will not intentionally increase exposure to environmental hazards [47].

Other Considerations

An important consideration for the RCT design is the timescale of the exposure–health relationship. It may be prohibitively expensive and difficult to mount RCTs for studying diseases with long latency periods, such as cancer or cardiovascular disease, but the RCT may be useful for testing the efficacy of interventions to reduce exposure to known carcinogens or cardiovascular risks (e.g. arsenic, lead) or for studying intermediate biological processes (e.g., systemic inflammation, endothelial dysfunction). The RCT design should also be considered in studies of acute or sub-chronic health effects and/or in situations with a well-defined exposure period of interest (e.g., pregnancy).

A key difference between clinical trials and RCTs of environmental interventions is that in the latter it is often difficult to blind participants and research personnel to intervention status. This may lead to biased effect estimates, particularly when investigating subjectively assessed health outcomes [48]. For example, unblinded studies of water treatment interventions in LMIC suggest that interventions produce substantial reductions in risk of diarrhea, while a smaller number of blinded studies have not found comparable benefits [49]. Even if participants cannot be blinded researchers should strive to rely on “hard,” objective outcomes, and personnel responsible for outcome assessment should be blinded (i.e., single blind).

Although many of the environmental health RCTs conducted to date have studied relatively simple interventions focused on a single environmental hazard, some have evaluated multifactorial interventions aimed at multiple exposures [50],[51]. Other fields have applied the RCT design more frequently and ambitiously. For example, the “Moving to Opportunities” study randomized over 4,000 families in high-poverty areas of several US cities into one of two housing mobility intervention groups or a control group and evaluated relationships with a range of outcomes [52][56]. The MIT Poverty Action Lab is using RCTs to address a wide range of questions in international development [57].

Our ongoing Ulaanbaatar Gestation and Air Pollution Research (UGAAR) study provides a useful example of a RCT in environmental health. Several large, well-conducted observational studies have suggested a link between ambient air pollution and impaired fetal growth [58],[59], but concerns about confounding and exposure misclassification remain [60]. We are randomizing approximately 500 pregnant women into either a HEPA filter intervention group or a control group (no filter); women in the intervention group will have HEPA filters operating in their home from enrollment until the child's birth. Ulaanbaatar, Mongolia is an ideal location to test the efficacy of this intervention because it has extraordinarily high air pollution concentrations [61] that are likely to remain elevated for decades even under the most optimistic scenarios [62],[63]. This situation is not unique—nearly 90% of the world's population breathes air pollution that exceeds WHO guidelines, and concentrations are increasing in much of the world [64] —so there is value in identifying efficacious interventions that can reduce health risks in the near term until regulations, technology, and economic development can reduce air pollution to acceptable levels. The randomized design should minimize confounding, and the high pollution concentrations and large exposure gradients created by the intervention provide statistical power among a relatively small study population on which detailed, household-level exposure assessment can be conducted. HEPA filters are a feasible and affordable intervention, and impaired fetal growth is a relevant outcome in this population.

Conclusions

Randomized controlled trials are standard practice in clinical and pharmaceutical research but have not been embraced by environmental health researchers. Greater use of the RCT design would complement the tremendous contributions made by other methods—including both observational epidemiology and toxicology—to our understanding of environmental risks and the development of environmental health policy. Researchers, academic institutions, and funding agencies have a role to play in expanding the use of RCTs in environmental health research. Researchers should think creatively about potential interventions and consider the RCT as a possible study design to test their specific research question. Funding agencies should allocate money specifically for randomized studies of environmental interventions. In addition to its scientific advantages, this would provide the additional benefit of encouraging research that aims not only to identify problems but also to identify possible solutions. Ethical issues must be considered carefully, and while institutional ethics approval is necessary, it is not sufficient to ensure that the research is conducted ethically. The RCT design has important limitations and is not applicable to all research questions, so observational studies will, and should, remain the workhorse in environmental health research. Nevertheless, RCTs can help advance the field of environmental health by creating new knowledge of exposure–health relationships, providing more definitive evidence of causality, identifying efficacious interventions to reduce or eliminate exposure and health risks, and countering the perception that environmental risks are evaluated with inadequate rigor.

Supporting Information

S1 Text.

Details of PubMed search for randomized controlled trials in environmental health.

https://doi.org/10.1371/journal.pmed.1001775.s001

(DOCX)

Author Contributions

Wrote the first draft of the manuscript: RWA PKB BPL. Wrote the paper: RWA PKB BPL. ICMJE criteria for authorship read and met: RWA PKB BPL. Agree with manuscript results and conclusions: RWA PKB BPL.

References

  1. 1. Grimes DA, Schulz KF (2012) False Alarms and Pseudo-Epidemics The Limitations of Observational Epidemiology. Obstet Gynecol 120: 920–927.
  2. 2. Taubes G (1995) Epidemiology Faces Its Limits. Science 269: 164–169.
  3. 3. Smith GD, Ebrahim S (2002) Data dredging, bias, or confounding. BMJ 325: 1437–1438.
  4. 4. Burnett RT, Pope CA, Ezzati M, Olives C, Lim SS, et al. (2014) An integrated risk function for estimating the global burden of disease attributable to ambient fine particulate matter exposure. Environ Health Perspect 122: 397–403.
  5. 5. Lanphear BP, Hornung R, Khoury J, Yolton K, Baghurstl P, et al. (2005) Low-level environmental lead exposure and children's intellectual function: An international pooled analysis. Environ Health Perspect 113: 894–899.
  6. 6. Darby S, Hill D, Auvinen A, Barrios-Dios JM, Baysson H, et al. (2005) Radon in homes and risk of lung cancer: collaborative analysis of individual data from 13 European case-control studies. BMJ 330: 223–226.
  7. 7. Chen Y, Graziano JH, Parvez F, Liu ML, Slavkovich V, et al. (2011) Arsenic exposure from drinking water and mortality from cardiovascular disease in Bangladesh: prospective cohort study. BMJ 342: 11.
  8. 8. Goodman M, Morgan RW, Ray R, Malloy CD, Zhao K (1999) Cancer in asbestos-exposed occupational cohorts: a meta-analysis. Cancer Causes Control 10: 453–465.
  9. 9. Lim SS, Vos T, Flaxman AD, Danaei G, Shibuya K, et al. (2012) A comparative risk assessment of burden of disease and injury attributable to 67 risk factors and risk factor clusters in 21 regions, 1990–2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet 380: 2224–2260.
  10. 10. Pruss-Ustun A, Corvalan C (2007) How much disease burden can be prevented by environmental interventions? Epidemiology 18: 167–178.
  11. 11. Pruss-Ustun A, Bonjour S, Corvalan C (2008) The impact of the environment on health by country: a meta-synthesis. Environ Health 7: 7.
  12. 12. Smith KR, Ezzati M (2005) How environmental health risks change with development: The epidemiologic and environmental risk transitions revisited. Annu Rev Environ Resour 30: 291–333.
  13. 13. Smith KR (2002) Indoor air pollution in developing countries: recommendations for research. Indoor Air 12: 198–207.
  14. 14. Chokshi DA, Farley TA (2012) The Cost-Effectiveness of Environmental Approaches to Disease Prevention. N Engl J Med 367: 295–297.
  15. 15. Gould E (2009) Childhood Lead Poisoning: Conservative Estimates of the Social and Economic Benefits of Lead Hazard Control. Environ Health Perspect 117: 1162–1167.
  16. 16. EPA (2011) The Benefits and Costs of the Clean Air Act from 1990 to 2020. Office of Air and Radiation. Available: http://www.epa.gov/cleanairactbenefits/feb11/summaryreport.pdf. Accessed 26 November 2014.
  17. 17. Hutton G, Rehfuess E, Tediosi F (2007) Evaluation of the costs and benefits of interventions to reduce indoor air pollution. Energy for Sustainable Development 11: 34–43.
  18. 18. Amitai Y, Brown MJ, Graef JW, Cosgrove E (1991) Residential deleading - effects on the blood lead levels of lead-poisoned children. Pediatrics 88: 893–897.
  19. 19. Aschengrau A, Beiser A, Bellinger D, Copenhafer D, Weitzman M (1997) Residential lead-based-paint hazard remediation and soil lead abatement: Their impact among children with mildly elevated blood lead levels. Am J Public Health 87: 1698–1702.
  20. 20. Woodcock A, Forster L, Matthews E, Martin J, Letley L, et al. (2003) Control of exposure to mite allergen and allergen-impermeable bed covers for adults with asthma. N Engl J Med 349: 225–236.
  21. 21. Prasad V, Vandross A, Toomey C, Cheung M, Rho J, et al. (2013) A Decade of Reversal: An Analysis of 146 Contradicted Medical Practices. Mayo Clin Proc 88: 790–798.
  22. 22. BC Centre for Disease Control (2014) Evidence Review: Using Masks to Protect Public Health During Wildfire Smoke Events. Available: http://www.bccdc.ca/NR/rdonlyres/5D1896FC-BE00-4DFB-B89A-39D4D98B8B82/0/WFSG_EvidenceReview_UsingMasks_FINAL_v5trs.pdf. Accessed 26 November 2014.
  23. 23. Patil SR, Arnold BF, Salvatore AL, Briceno B, Ganguly S, et al. (2014) The Effect of India's Total Sanitation Campaign on Defecation Behaviors and Child Health in Rural Madhya Pradesh: A Cluster Randomized Controlled Trial. PLoS Med 11: 17.
  24. 24. Clancy L, Goodman P, Sinclair H, Dockery DW (2002) Effect of air-pollution control on death rates in Dublin, Ireland: an intervention study. Lancet 360: 1210–1214.
  25. 25. Pope CA (1989) RESPIRATORY-DISEASE ASSOCIATED WITH COMMUNITY AIR-POLLUTION AND A STEEL MILL, UTAH VALLEY. Am J Public Health 79: 623–628.
  26. 26. Johnston FH, Hanigan IC, Henderson SB, Morgan GG (2013) Evaluation of interventions to reduce air pollution from biomass smoke on mortality in Launceston, Australia: retrospective analysis of daily mortality, 1994–2007. BMJ 346: 11.
  27. 27. Pirkle JL, Brody DJ, Gunter EW, Kramer RA, Paschal DC, et al. (1994) The Decline In Blood Lead Levels In The United-States - The National-Health And Nutrition Examination Surveys (Nhanes). JAMA 272: 284–291.
  28. 28. Groves-Kirkby CJ, Denman AR, Phillips PS, Crockett RGM, Woolridge AC, et al. (2006) Radon mitigation in domestic properties and its health implications - a comparison between during-construction and post-construction radon reduction. Environment International 32: 435–443.
  29. 29. Henderson DE, Milford JB, Miller SL (2005) Prescribed burns and wildfires in Colorado: Impacts of mitigation measures on indoor air particulate matter. J Air Waste Manag Assoc 55: 1516–1526.
  30. 30. Fritz JA, Cleland J (2003) Effectiveness versus efficacy: More than a debate over language. J Orthop Sports Phys Ther 33: 163–165.
  31. 31. Rothwell PM (2006) Factors that can affect the external validity of randomized controlled trials. PLoS Clin Trials 1: e9.
  32. 32. Schmidt WP, Cairncross S (2009) Household Water Treatment in Poor Populations: Is There Enough Evidence for Scaling up Now? Environ Sci Technol 43: 986–992.
  33. 33. Rehfuess EA, Bartram J (2014) Beyond direct impact: Evidence synthesis towards a better understanding of effectiveness of environmental health interventions. Int J Hyg Environ Health 217: 155–159.
  34. 34. Kansagra SM, Farley TA (2011) Public Health Research: Lost in Translation or Speaking the Wrong Language? Am J Public Health 101: 2203–2206.
  35. 35. Rose G (2001) Sick individuals and sick populations. Int J Epidemiol 30: 427–432.
  36. 36. Brauner EV, Forchhammer L, Moller P, Barregard L, Gunnarsen L, et al. (2008) Indoor particles affect vascular function in the aged - An air filtration-based intervention study. Am J Respir Crit Care Med 177: 419–425.
  37. 37. Lanphear BP, Hornung RW, Khoury J, Yolton K, Lierl M, et al. (2011) Effects of HEPA Air Cleaners on Unscheduled Asthma Visits and Asthma Symptoms for Children Exposed to Secondhand Tobacco Smoke. Pediatrics 127: 93–101.
  38. 38. Allen RW, Carlsten C, Karlen B, Leckie S, van Eeden S, et al. (2011) An Air Filter Intervention Study of Endothelial Function among Healthy Adults in a Woodsmoke-impacted Community. Am J Respir Crit Care Med 183: 1222–1230.
  39. 39. Resnik DB (2008) Randomized controlled trials in environmental health research: Ethical issues. J Environ Health 70: 28–30.
  40. 40. Freedman B (1987) Equipoise And The Ethics Of Clinical Research. N Engl J Med 317: 141–145.
  41. 41. Angell M (1990) The Nazi Hypothermia Experiments And Unethical Research Today. N Engl J Med 322: 1462–1464.
  42. 42. Kumar A, Soares H, Wells R, Clarke M, Hozo L, et al. (2005) Are experimental treatments for cancer in children superior to established treatments? Observational study of randomised controlled trials by the Children's Oncology Group. BMJ 331: 1295–1298B.
  43. 43. Sathyanarayana S, Alcedo G, Saelens BE, Zhou C, Dills RL, et al. (2013) Unexpected results in a randomized dietary trial to reduce phthalate and bisphenol A exposures. J Expo Sci Environ Epidemiol 23: 378–384.
  44. 44. Emanuel EJ, Wendler D, Grady C (2000) What makes clinical research ethical? JAMA 283: 2701–2711.
  45. 45. El Setouhy M, Agbenyega T, Anto F, Clerk CA, Koram KA, et al. (2002) Fair benefits for research in developing countries. Science 298: 2133–2134.
  46. 46. Glickman SW, McHutchison JG, Peterson ED, Cairns CB, Harrington RA, et al. (2009) Ethical and Scientific Implications of the Globalization of Clinical Research. N Engl J Med 360: 816–823.
  47. 47. Resnik DB, Wing S (2007) Lessons learned from the children's environmental exposure research study. Am J Public Health 97: 414–418.
  48. 48. Wood L, Egger M, Gluud LL, Schulz KF, Juni P, et al. (2008) Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ 336: 601–605.
  49. 49. Clasen T, Roberts I, Rabie T, Schmidt W, Cairncross S (2006) Interventions to improve water quality for preventing diarrhoea. Cochrane Database Syst Rev (3): CD004794.
  50. 50. Morgan WJ, Crain EF, Gruchalla RS, O'Connor GT, Kattan M, et al. (2004) Results of a home-based environmental intervention among urban children with asthma. N Engl J Med 351: 1068–1080.
  51. 51. Hartinger SM, Lanata CF, Hattendorf J, Gil AI, Verastegui H, et al. (2011) A community randomised controlled trial evaluating a home-based environmental intervention package of improved stoves, solar water disinfection and kitchen sinks in rural Peru: Rationale, trial design and baseline findings. Contemp Clin Trials 32: 864–873.
  52. 52. Kessler RC, Duncan GJ, Gennetian LA, Katz LF, Kling JR, et al. (2014) Associations of Housing Mobility Interventions for Children in High-Poverty Neighborhoods With Subsequent Mental Disorders During Adolescence. JAMA 311: 937–947.
  53. 53. Ludwig J, Sanbonmatsu L, Gennetian L, Adam E, Duncan GJ, et al. (2011) Neighborhoods, Obesity, and Diabetes - A Randomized Social Experiment. N Engl J Med 365: 1509–1519.
  54. 54. Leventhal T, Brooks-Gunn J (2004) A randomized study of neighborhood effects on low-income children's educational outcomes. Dev Psychol 40: 488–507.
  55. 55. Ludwig J, Duncan GJ, Gennetian LA, Katz LF, Kessler RC, et al. (2012) Neighborhood Effects on the Long-Term Well-Being of Low-Income Adults. Science 337: 1505–1510.
  56. 56. Sciandra M, Sanbonmatsu L, Duncan GJ, Gennetian LA, Katz LF, et al. (2013) Long-term effects of the Moving to Opportunity residential mobility experiment on crime and delinquency. J Exp Criminol 9: 451–489.
  57. 57. J-PAL (2005) Abdul Latif Jameel Poverty Action Lab. Available: http://www.povertyactionlab.org/. Accessed 28 October 2014.
  58. 58. Dadvand P, Parker J, Bell ML, Bonzini M, Brauer M, et al. (2013) Maternal Exposure to Particulate Air Pollution and Term Birth Weight: A Multi-Country Evaluation of Effect and Heterogeneity. Environ Health Perspect 121: 367–373.
  59. 59. Savitz DA, Bobb JF, Carr JL, Clougherty JE, Dominici F, et al. (2014) Ambient Fine Particulate Matter, Nitrogen Dioxide, and Term Birth Weight in New York, New York. Am J Epidemiol 179: 457–466.
  60. 60. Bosetti C, Nieuwenhuijsen MJ, Gallus S, Cipriani S, La Vecchia C, et al. (2010) Ambient particulate matter and preterm birth or birth weight: a review of the literature. Arch Toxicol 84: 447–460.
  61. 61. Allen RW, Gombojav E, Barkhasragchaa B, Byambaa T, Lkhasuren O, et al.. (2011) An assessment of air pollution and its attributable mortality in Ulaanbaatar, Mongolia. Air Qual Atmos Health. E-pub ahead of print. doi:10.1007/s11869-011-0154-3
  62. 62. Fenger J (1999) Urban air quality. Atmos Environ 33: 4877–4900.
  63. 63. Ochir C, Smith KR, Hill LD, Baljinnyam P, Damdinsuren Y, et al.. (2014) Air Pollution and Health in Ulaanbaatar. Available: http://ehs.sph.berkeley.edu/krsmith/publications/2014/UB%20Final%20Report%20-%20July%2010.pdf. Accessed 26 November 2014.
  64. 64. Brauer M, Amann M, Burnett RT, Cohen A, Dentener F, et al. (2012) Exposure Assessment for Estimation of the Global Burden of Disease Attributable to Outdoor Air Pollution. Environ Sci Technol 46: 652–660.