Whether decisions about the health care of children are made by parents, carers, health practitioners, or children themselves, ideally these decisions will be informed by the results of systematic reviews of previous clinical trials. Unfortunately, drawing on research from the past when making these decisions is not straightforward, even with the advent of systematic reviews, because different trials have used different outcome measures. Indeed, systematic reviews have helped to highlight the problems that arise when researchers from different times and places—sometimes even in the same times and places—use a variety of health outcomes and outcome measures to assess the effect of interventions [1].

In a new systematic review published in this issue of PLoS Medicine, Ian Sinha and colleagues examined studies that involved the selection of outcomes for use in paediatric clinical trials [2]. As they say in their report, the choice of outcome is important because choosing inappropriate outcomes may lead to “wasted resources or misleading information which either overestimates, underestimates or completely misses the potential benefits of an intervention.”

Setting the Scene

In trying to decide whether the work done by Sinha and colleagues is important, a useful starting point is knowing just how many clinical trials have been conducted in children. Although it is difficult to find a reliable figure, a search of the Cochrane Central Register of Controlled Trials, the world's single largest repository of records for randomised trials, finds more than 30,000 reports simply on the basis of having the words “child” or “children” in either their title or abstract. A PubMed search, limiting the records to all reports about humans, finds a total of just over 10 million articles. Further limiting this search to records for “clinical trials” brings this figure down to about 520,000, and limiting this again to “children” found 127,904 (search performed March 17, 2008). If you try again now, one month later, you'll get a sense of how fast things are moving—even in one month it is likely that additional child health trials have been published. Moreover, anyone trying to compare and contrast different paediatric trials will be faced by a wide variety of outcomes reported and a wide variety of ways in which the same outcomes were measured.

Trial Outcome Measures for Children

Sinha and colleagues set out to find reports of studies done to determine or agree upon a standard set of outcome measures for clinical trials involving children. Doing a systematic review helped them to minimise bias in making decisions about how to search the literature, what to include and exclude, and how to report their findings. Their initial wide search retrieved nearly 9,000 abstracts, which led to 70 articles being evaluated in full, leading ultimately to 25 eligible articles from 13 collaborative groups. (Figure 2 in [2] shows the flow of papers through the systematic review process.)

Perhaps the most consistent, and alarming, finding across the 25 studies was that none of them involved children in the process of determining the selection of outcomes, although the authors note the potential challenges of doing so. There were 13 paediatric conditions, including asthma, Crohn's disease, and cystic fibrosis, as well as a number of clinical symptoms or signs, such as pain and apnoea, for which research has been done to determine which outcomes should be measured in trials.

Implications of the New Study

Reaching agreement on standard outcomes for clinical trials in children is important to enable researchers and clinicians to compare, contrast, and combine the findings of these trials. Such agreement should also help to reduce the possibility of biased reporting. If researchers have a choice of outcomes, they might be tempted to report selectively, emphasising those that give them the results they find most attractive and/or omitting those that are unfavourable to their point of view [3]. When there is a standard set of outcomes and outcome measures, users of the results of trials should be rightly cautious of any trials that do not report all of these outcomes.

Next Steps

Sinha and colleagues identified 13 groups who have tried to tackle the problem of poor standardisation in the outcomes of clinical trials in children. These groups are not alone among those seeking to standardise trial outcomes. In the early 1990s, the Outcome Measures in Rheumatology Group (OMERACT; see http://www.omeract.org/) began its efforts to produce standardisation in assessments of the effects of treatments for rheumatoid arthritis [4], and a similar process was reported recently for ulcerative colitis [5]. The success of OMERACT probably owes much to its attempts to reach a consensus among major stakeholders. Recommendations for chronic pain research, developed by a group called the Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials (IMMPACT; http://www.immpact.org/), followed the OMERACT model [6]. An electronic Delphi survey, as used by several of the groups working in child health, was also used to prepare a core set of outcomes for maternity care [7].

Making trials from the past and present more useful to decision makers in child health in the future will require uptake of the recommendations proposed in studies such as those found by Sinha and colleagues. But the challenges of achieving standardised outcomes in clinical trials are formidable. These challenges are well demonstrated in research in adults with schizophrenia. In the 1990s, 2,000 trials of schizophrenia were found to have used 640 different rating scales in their assessments of 600 interventions [8]. A subsequent systematic review of chlorpromazine versus placebo for schizophrenia noted: “If rating scales are to be employed, a concerted effort should be made to agree on which measures are the most useful. Studies within this review reported on so many scales that, even if results had not been poorly reported, they would have been difficult to synthesise in a clinically meaningful way” [9]. A similar lack of standardisation in trial outcomes has been noted for research on asthma and the common cold. In their Cochrane systematic review of inhaled magnesium sulphate for asthma, Blitz and colleagues wrote that “There is a strong argument for asthma researchers to develop a consensus regarding the reporting of pulmonary function results” [10]. Similarly, in their systematic review of echinacea for the common cold, Linde and colleagues noted that “It would be desirable if experts in research on common colds would develop recommendations for a core set of outcome measures to be used and reported in randomized clinical trials” [11].

Conclusion

Sinha and colleagues have produced what is probably the most comprehensive account to date of efforts to standardise outcomes in clinical trials in children. As the authors say, referring to the studies they found, “When designing clinical trials in these conditions, this work should make the selection of outcomes easier and more uniform.” I would add that when trying to choose outcomes for clinical trials in any condition, Sinha and colleagues' work should make it easier to plan, find, appraise, and use initiatives that have already attempted to standardise these outcomes.