Citation: Ioannidis JPA, Garber AM (2011) Individualized Cost-Effectiveness Analysis. PLoS Med 8(7): e1001058. doi:10.1371/journal.pmed.1001058
Published: July 12, 2011
Copyright: © 2011 Ioannidis, Garber. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: No funding was received for this work.
Competing interests: JI is a member of the PLoS Medicine Editorial Board. The authors have declared that no other competing interests exist.
Abbreviations: ICER, incremental cost-effectiveness ratio; QALD, quality-adjusted life day; QALY, quality-adjusted life year
Provenance: Not commissioned; externally peer reviewed
- Cost-effectiveness analyses typically express their principal results as incremental cost-effectiveness ratios (ICERs).
- ICERs are useful in making decisions for allocation of resources at a population level, but typical ICER measures have shortcomings when used for individual decisions.
- For the same ICER, the cost-effectiveness may vary among individuals because not everyone assigns the same priorities to specific outcomes, shares the same attitudes toward risk, or faces the same distribution of expected outcomes.
- ICER information can be enhanced by providing additional metrics that individualize cost-effectiveness analyses.
- These metrics include the per person net benefit and cost, subgroup ICER estimates for observed measured sources of heterogeneity, and distributions of outcomes and costs for unknown or unmeasured sources of heterogeneity.
The results of a typical cost-effectiveness (cost–utility) analysis are expressed by the incremental cost-effectiveness ratio (ICER) : the money required to gain a quality-adjusted life year (QALY) (i.e., one year with best possible quality of life) at a population level. The ICER concept is valuable for choosing among diverse interventions competing for limited resources. It was developed primarily for societal or group-level decisions, such as allocations of a fixed governmental health budget, and consequently is particularly helpful for health authorities and other decision-makers who wish to prioritize resource allocation in health care to numerous interventions on diverse diseases across whole systems. The major contribution of ICER to inform population-level decisions is even more obvious today, as health care costs are escalating and rational choices need to be made on how to contain cost without compromising health outcomes at the societal level. However, conventional ICERs are population-level tools, and fail to take into account important inter-individual differences that might affect the value of a particular intervention. The choice that maximizes the population's health or has the best ICER overall is not always the same as the best choice for a specific individual. Moreover, the best choices may differ for different individuals. There is thus interest in how to modify the ICER concept for applications in individual decision-making ,. In this essay, we aim to contribute to the discussion on how to use ICER and related metrics in a way that would be more useful for decision-making at the individual level, whether used by clinicians or individual patients.
The Concept of Individualized Cost-Effectiveness and Individual Choices
By convention, the numerator of the ICER is the difference in cost of care between compared interventions. The denominator is the corresponding difference in health outcomes (usually measured in QALYs). The denominator combines disparate kinds of information: diverse health benefits and harms are summed into a net health outcome measure.
As typically measured, the ICER appraises the average experience with an intervention. However, several investigators have pointed out that working with averages is not good enough. According to Kravitz et al. , “averages do not apply to everyone.” As is now well-recognized, averages are problematic for interpreting clinical evidence, e.g., the results of randomized trials, where treatment effects (benefits, but also harms) are often heterogeneous, i.e., different for different types of patients –. The same challenges arise when one considers cost-effectiveness and decision-making. After all, randomized trials and other clinical studies typically feed their data into decision and cost-effectiveness models. Moreover, the granularity of the cost and outcomes per patient may be important to convey, i.e., specifying separately the different outcomes, so that clinicians and patients may be better informed and able to make better choices.
Sometimes there is no compelling reason to distinguish different outcomes rather than using a summary measure, especially if all the outcomes are similar in severity and consequences. For example, consider two medicines that treat the common cold. One of them reduces the average duration of the cold by three days, while the other reduces the average duration by four days, but on average the second one also causes one day of headache as a side effect. As it happens, the headaches are no better or worse than having a cold. From the patient's point of view, both drugs decrease the duration of symptomatic illness by three days, with no meaningful difference between them.
However, lumping outcomes can be problematic when two interventions, even if they have identical mean effects, have very different distributions of outcomes. For example, suppose one intervention reduces the duration of the cold by an extra day relative to the other treatment, but on rare occasions it causes side effects that are either severe or prolonged, so the net effect on health is identical, on average. Then, some patients might reasonably decide that they would prefer the drug that is less effective most of the time, rather than risk the small possibility of becoming very ill if they were unlucky enough to experience the major side effect.
Heterogeneity in outcomes and costs across patients may stem from both observed (known) and unobserved (unknown or unmeasured) sources. The former arise from variation in well-known, validated, measured characteristics that predictably influence outcomes or costs. For example, the benefit of an antibiotic and the cost of treating an infection might be well known to vary with the severity of the infection or the sensitivity of the causative organism. Heterogeneity from unobservable sources essentially reflects the variation in outcomes and costs that occurs within a group of individuals with otherwise identical observable characteristics. This variation is due to chance or due to predictors that are not discovered yet or are not measured.
Situations Where the Traditional ICER Alone Is Insufficient
Table 1 lists situations where even though the ICER is the same, the individualized cost-effectiveness (and the resulting choices) may vary. Note that this distinction between requirements for individual decision-making and population-level ICERs is not a reflection of the typical perspective question, i.e., whose costs should be considered in the analysis (the society's, the payer's, the insurer's, and so forth). We assume here a societal perspective or the perspective of the perfect insurer in the calculation of all ICER estimates, as is standard practice in cost-effectiveness analysis .
Table 1. Some situations where the traditional ICER does not necessarily suffice for individualized decision-making.doi:10.1371/journal.pmed.1001058.t001
First, when the ICER is the same for two treatments for different conditions, each intervention may seem to be an equally good investment. However, a patient may have strong preferences for one over the other. For example, consider one intervention that improves outcomes in a few very sick patients who would die otherwise, but can live good-quality lives with treatment; both the benefits and the costs are very large. Another intervention has an equally favorable ICER, but it confers very small benefits to many individuals who are not very sick at baseline, with little cost per person. In many circumstances, the former intervention would be favored. Economists recognize that individuals place a greater value for the same absolute gains in life expectancy as life expectancy becomes shorter, all else being equal. For example, cadaveric donor liver transplantation has an ICER of US$42,000/QALY  and in many respects is more widely adopted than intensive lifestyle intervention in adults with impaired glucose tolerance, which has at least as favorable an ICER (one analysis gives even an ICER of US$11,000/QALY ). Most of the time, selecting between interventions for different conditions is a challenge that policy-makers, not individuals, face. However, individuals can face similar situations, particularly at different times in their lives.
Second, two different interventions for the same condition may have the same ICER, and the same expected outcomes, but the distributions of health outcomes might differ—one might have greater variance than the other. Some people may be more risk-averse than others –. Risk-averse people would sometimes prefer an intervention that provided inferior outcomes, on average, to an alternative that also had greater variance in outcomes. For example, suppose that two surgical interventions have the same ICER of $25,000/QALY. One operation costs $50,000 and confers 2.1 QALYs of benefit against 0.1 QALYs of harms due to a manageable problem that stems from the surgery. Another more aggressive procedure also costs $50,000 but confers 2.3 QALYs of benefit and, on average, 0.3 QALY of expected harm: 99% of the patients get no harm, but 1% die during the surgery and lose 30 years of life. Even though both operations have the same ICER, risk-averse patients may prefer to avoid the aggressive surgery.
Finally, the best choice may vary with an individual's background health and other, often highly personal, circumstances. Preferences over length of life may not be a simple function of its length. For example, a patient with late-stage metastatic cancer may wish to live until a landmark date, e.g., her daughter's wedding. The value of additional days of life may decline sharply after the landmark event. Among interventions with the same ICER, the patient will choose the one that maximizes the chances of remaining alive until the wedding.
These limitations of conventional cost-effectiveness analyses have long been recognized, yet the focus on ICER values averaged across a population is widespread. Effective communication of individualized analyses may require alternative approaches to presenting cost-effectiveness information.
Possible Ways to Present Individualized Cost-Effectiveness
Box 1 lists different approaches to individualizing cost-effectiveness.
Box 1. Approaches to Presenting Individualized Cost-Effectiveness Analysis
- Per person benefit in QALYs or quality adjusted life days
- Per person cost
- Subgroup-specific estimates of per person benefit and per person cost for well-documented and validated subgroups based on known sources of heterogeneity
- Distribution (extent of variance) of benefits overall and in subgroup-specific estimates
- Distribution (extent of variance) in costs overall and in subgroup-specific estimates
Per Person Benefit and Cost
The reporting of the expected cost and QALY gain or loss per person is already widely considered standard practice for a good cost-effectiveness analysis. However, even recent papers don't always report these numbers transparently. To illustrate how this works, suppose an analysis simulates costs and expected outcomes for 1,000,000 patients. Eventually, the analysis shows that 500 QALYs are gained by the new intervention versus a comparator at an incremental cost of $5,000,000. The ICER is $5,000,000/500 = $10,000/QALY. The per person cost is $5, for a gain of 0.0005 QALYs.
Table 2 shows the traditional ICER and calculated per person information from eight examples of recent cost-effectiveness analyses –. The individualized information is often easier to understand when the QALYs are expressed in days, i.e., in quality-adjusted life days (QALDs), because for many interventions the increase in QALYs is much less than one QALY. As shown, the absolute magnitudes of the net benefit and of the cost per person vary enormously across interventions, far more than the variation in the ICER. The average QALYs (or QALDs) gained per person varies from 0.18 days for screening for postnatal depression to 223 days for laparoscopic fundoplication for gasto-esophageal reflux, a more than 1,200-fold difference. The cost per person varies from about US$9 for rotavirus vaccination to US$2,586 for the fundoplication, an almost 300-fold difference. Conversely, in these examples, differences in traditional ICER are much smaller.
Table 2. Examples of ICERs and calculated per person benefits and costs.doi:10.1371/journal.pmed.1001058.t002
Observed Sources of Heterogeneity
Average results can be misleading for many patients ,. Whenever possible, information should be tailored to identifiable subgroups with characteristics that explain variability in outcomes and costs. Costs and outcomes should be presented for groups with different absolute benefits (even if relative benefits, or relative risk reduction from an intervention, are identical). Frequently authors are justifiably reluctant to do so, particularly when the information is derived from post-hoc subgroup analyses of trial data. The extent of desirable subgrouping depends on the available evidence. There is a tipping point at which stratifying outcomes too finely and with limited evidence becomes cumbersome and introduces too much uncertainty to be useful for making decisions. However, when well-validated prognostic models are available or studies using sufficient independent data support the specific subgroup classifications ,, cost–utility information can be confidently individualized by level of predicted risk and subgroup. For example, a cost-effectiveness analysis of tissue plasminogen activator versus streptokinase for thrombolysis in acute myocardial infarction has presented subgroup ICER estimates for different locations of infarction and different age groups, with up to 16-fold differences across these subgroups .
Unobserved Sources of Heterogeneity
When heterogeneity in outcomes or costs arises from unknown sources, every patient who receives an intervention initially faces the same distribution of potential outcomes. When heterogeneity matters, information on the distribution of potential outcomes and costs should also be presented, even though we can't predict which patient will get the most benefit, and who will get the rare but devastating adverse event. An illustrative example of how to present this information appears in Table 3.
Table 3. Hypothetical example of different individual experiences for patients who do and do not experience different events (benefit, harm, major harm, or other rare events).doi:10.1371/journal.pmed.1001058.t003
The optimal way to present distributions depends on the type of variable. For example, most continuous measures (including cost) can be summarized with means/medians and standard deviations or interquartile ranges. Time-to-event outcomes such as long-term survival are conveniently represented by Kaplan-Meier plots. Utility or quality weights can also be plotted over time, with expected mean values and variances thereof. Presenting distributions in full detail requires space, and their interpretation can be burdensome, but both problems can be addressed, e.g., by online supplements that include detailed explanatory material.
Societal versus Patient Perspective
Thus far we have assumed a societal perspective for the calculation of costs. We should acknowledge that if patients don't have to pay for any of the health care costs themselves, they may opt for the choice that maximizes their net benefits regardless of the cost. However, in many health systems, cost is taken into account and limits the choices of individuals.
Even in countries with national health systems that have minimized the role of individual cost-sharing, individuals pay growing shares of the costs of health and health-related services out-of-pocket. Direct-to-consumer advertisement, over-the-counter interventions, and personalized (e.g., genomic) testing are examples of growing markets that try to appeal to individuals who pay for services with their own funds –. Because patients spending their own money have a compelling interest in the value of the services they consume, cost-effectiveness analysis is not solely the province of government or corporate decision-makers. Whenever costs and outcomes from the patient perspective differ from those from the population/societal perspective, costs and outcomes should be reported from both perspectives. This will greatly increase the range of usefulness of the analysis. Across different fields of economics, consumer demand is an important consideration. It is possible that policy-makers use ICER to make decisions currently without information that patients would find useful.
Moreover, cost-effectiveness analysis has relevance to patients, even if they do not bear costs directly, and to physicians with an interest in ensuring the wise use of resources. The costs of care are typically hidden or unknown to patients and physicians. Similarly, there may be a poor understanding of what outcomes are really at stake for an individual. Further studies are needed to clarify whether provision of individualized information influences the behavior and choices of physicians and patients.
Once cost, benefits, and harms are presented separately for different types of patients, one can take an extra step and directly incorporate personal preferences in the calculations, i.e., placing different weights on the same side effects and benefits of therapy for different patients ,. One may even assign different values to different time sequences of beneficial and harmful events, e.g., by reporting the healthy-years equivalents of lifetime health profiles –. However, personal weights for each type of cost and outcome and healthy-years equivalents for each of many possible complex lifetime health profiles are often difficult to elicit.
Individualized analysis is not always necessary or appropriate. First, the individualized cost-effectiveness information is directly meaningful only for interventions and strategies that are applied to a single person and where the beneficial or harmful impacts of the intervention do not extend beyond the specific type of individual. For example, full individualization cannot be applied to a cost-effectiveness analysis of modernization of adult critical care services , where the intervention is made at the level of a population service. Also, in cost-effectiveness analyses of vaccination of boys with human papilloma virus vaccine  or of influenza vaccination strategies , the intended benefits extend also to the partners or contacts, respectively, of the vaccinated individuals. In addition to the personal stake for the vaccinated individuals, implications for the health of other people may be substantial. Sometimes, the cumulative impact on other people's health may far exceed the impact on the single individual who gets such interventions.
Furthermore, estimates of individualized cost-effectiveness, much like traditional ICER estimates, are subject to errors in the assumptions that feed into their calculations. These errors may stem from a lack of evidence, poor representation of the evidence, or even manipulations due to conflicts of interest –. Erroneous inferences may result from unrealistic modeling of the decision process or unrealistic estimates for the various benefits, harms, and costs. Moreover, uncertainty in these values  or even in risk aversion  should not be underestimated. Sensitivity analyses incorporating a range of values for key parameters can be applied to the individualized cost-effectiveness, much as they are applied to the traditional ICER.
Cost-effectiveness analysis offers a foundation for rational decision-making and can be very helpful in making health care more efficient and effective at the population level . Such analyses can often be more useful for clinicians and for individual patients as well, when they individualize the cost–utility information they present. Individual-tailored information can complement the traditional ICER.
Conceived and designed the experiments: JIAG. Analyzed the data: JI AG. Wrote the first draft: JI. Contributed to the writing of the manuscript: JI AG. ICMJE criteria for authorship read and met: JI AG. Agree with the manuscript's results and conclusions: JI AG.
- 1. Gold MR, Siegel JE, Russell LB, Weinstein MC, editors. (1996) Cost-effectiveness in health and medicine. Oxford: Oxford University Press.
- 2. Willan AR (2001) On the probability of cost-effectiveness using data from randomized clinical trials. BMC Med Res Methodol 1: 8. doi: 10.1186/1471-2288-1-8
- 3. O'Hagan A, Stevens JW (2002) The probability of cost-effectiveness. BMC Med Res Methodol 2: 5. doi: 10.1186/1471-2288-2-5
- 4. Kravitz RL, Duan N, Braslow J (2004) Evidence-based medicine, heterogeneity of treatment effects, and the trouble with averages. Milbank Q 82: 661–687. doi: 10.1111/j.0887-378X.2004.00327.x
- 5. McLaughlin MJ, HTE Policy Roundtable Panel (2007) Healthcare policy implications of heterogeneity of treatment effects. Am J Med 120(4): Suppl 1S32–S35. doi: 10.1016/j.amjmed.2007.02.006
- 6. Greenfield S, Kravitz R, Duan N, Kaplan SH (2007) Heterogeneity of treatment effects: implications for guidelines, payment, and quality assessment. Am J Med 120(4): Suppl 1S3–S9. doi: 10.1016/j.amjmed.2007.02.002
- 7. Glasziou PP, Irwig LM (1995) An evidence based approach to individualising treatment. BMJ 311: 1356–1359. doi: 10.1136/bmj.311.7016.1356
- 8. Garber AM (2000) Advances in cost-effectiveness analysis of health interventions. In: Culyer AJ, Newhouse JP, editors. Handbook of health economics, Volume 1. Amsterdam: North-Holland. pp. 181–221.
- 9. Mendeloff J, Ko K, Roberts MS, Byrne M, Dew MA (2004) Procuring organ donors as a health investment: how much should we be willing to spend? Transplanation 78: 1704–1710. doi: 10.1097/01.tp.0000149787.97288.a2
- 10. Herman WH, Hoerger TJ, Brandle M, Hicks K, Sorensen S, et al. (2005) The cost-effectiveness of lifestyle modification or metformin in preventing type 2 diabetes in adults with impaired glucose tolerance. Ann Intern Med 142: 323–332. doi: 10.7326/0003-4819-142-5-200503010-00007
- 11. Luce R (2000) Utility of gains and losses: measurement-theoretical and experimental approaches. Mahwah (New Jersey): Lawrence Erlbaum Associates.
- 12. Byrnes JP, Miller DC, Schafer WD (1999) Gender differences in risk taking: A meta-analysis. Psych Bull 125: 367–383. doi: 10.1037//0033-2909.125.3.367
- 13. Elbasha EH (2005) Risk aversion and uncertainty in cost-effectiveness analysis: the expected-utility, moment-generating function approach. Health Econ 14: 457–470. doi: 10.1002/hec.915
- 14. Sapienza P, Zingales L, Maestripieri D (2009) Gender differences in financial risk aversion and career choices are affected by testosterone. Proc Natl Acad Sci U S A 106: 15268–15273. doi: 10.1073/pnas.0907352106
- 15. Finckh A, Bansback N, Marra CA, Anis AH, Michaud K, et al. (2009) Treatment of very early rheumatoid arthritis with symptomatic therapy, disease-modifying antirheumatic drugs, or biologic agents: a cost-effectiveness analysis. Ann Intern Med 151: 612–621. doi: 10.7326/0003-4819-151-9-200911030-00006
- 16. Eckman MH, Rosand J, Greenberg SM, Gage BF (2009) Cost-effectiveness of using pharmacogenetic information in warfarin dosing for patients with nonvalvular atrial fibrillation. Ann Intern Med 150: 73–83. doi: 10.7326/0003-4819-150-2-200901200-00005
- 17. Pletcher MJ, Lazar L, Bibbins-domingo K, Moran A, Rodondi N, et al. (2009) Comparing impact and cost-effectiveness of primary prevention strategies for lipid lowering. Ann Intern Med 150: 243–254. doi: 10.7326/0003-4819-150-4-200902170-00005
- 18. Paulden M, Palmer S, Hewitt C, Gilbody S (2009) Screening for postnatal depression in primary care: cost effectiveness analysis. BMJ 339: b5203. doi:10.1136/bmj.b5203.
- 19. Latimer N, Lord J, Grant RL, O'Mahony R, Dickson J, et al. (2009) Cost effectiveness of COX 2 selective inhibitors and traditional NSAIDs alone or in combination with a proton pump inhibitor for people with osteoarthritis. BMJ 339: b2538. doi:10.1136/bmj.b2538.
- 20. Epstein D, Bojke L, Sculpher MJ, REFLUX Trial Group (2009) Laparoscopic fundoplication compared with medical management for gastro-oesophageal reflux disease: cost effectiveness study. BMJ 339: b2576. doi:10.1136/bmj.b2576.
- 21. Rose J, Hawthorn RL, Watts B, Singer ME (2009) Public health impact and cost effectiveness of mass vaccination with live attenuated human rotavirus vaccine (RIX4414) in India: model based analysis. BMJ 339: b3653. doi:10.1136/bmj.b3653.
- 22. Barton GR, Sach TH, Jenkinson C, Doherty M, Avery AJ, et al. (2009) Lifestyle interventions for knee pain in overweight and obese adults aged > or = 45: economic evaluation of randomised controlled trial. BMJ 339: b2273. doi: 10.1136/bmj.b2273
- 23. Laska EM, Meisner M, Siegel C (1997) Statistical inference for cost-effectiveness ratios. Health Econ 6: 229–242. doi: 10.1002/(SICI)1099-1050(199705)6:3<229::AID-HEC268>3.0.CO;2-M
- 24. Briggs A, Fenn P (1997) Trying to do better than average: a commentary on ‘statistical inference for cost-effectiveness ratios’. Health Econ 6: 491–495. doi: 10.1002/(SICI)1099-1050(199709)6:5<491::AID-HEC293>3.0.CO;2-R
- 25. Ioannidis JP, Lau J (1997) The impact of high-risk patients on the results of clinical trials. J Clin Epidemiol 50: 1089–1098. doi: 10.1016/S0895-4356(97)00149-2
- 26. Kent DM, Rothwell PM, Ioannidis JP, Altman DG, Hayward RA (2010) Assessing and reporting heterogeneity in treatment effects in clinical trials: a proposal. Trials 11: 85. doi: 10.1186/1745-6215-11-85
- 27. Mark DB, Hlatky MA, Califf RM, Naylor CD, Lee KL, et al. (1995) Cost effectiveness of thrombolytic therapy with tissue plasminogen activator as compared with streptokinase for acute myocardial infarction. N Engl J Med 332: 1418–1424. doi: 10.1056/NEJM199505253322106
- 28. Gilbody S, Wilson P, Watt I (2005) Benefits and harms of direct to consumer advertising: a systematic review. Qual Saf Health Care 14: 246–250. doi: 10.1136/qshc.2004.012781
- 29. Wright CF, Hall A, Zimmern RL (2011) Regulating direct-to-consumer genetic tests: What is all the fuss about? Genet Med 13: 295–300. doi: 10.1097/GIM.0b013e3181f69dd2
- 30. Cohen JP, Paquette C, Cairns CP (2005) Switching prescription drugs to over the counter. BMJ 330: 39–41. doi: 10.1136/bmj.330.7481.39
- 31. Nease RF Jr, Owens DK (1994) A method for estimating the cost-effectiveness of incorporating patient preferences into practice guidelines. Med Decis Making 14: 382–392. doi: 10.1177/0272989X9401400409
- 32. Gage BF, Cardinalli AB, Owens DK (1998) Cost-effectiveness of preference-based antithrombotic therapy for patients with nonvalvular atrial fibrillation. Stroke 29: 1083–1091. doi: 10.1161/01.STR.29.6.1083
- 33. Gafni A, Birch S, Mehrez A (1993) Economics, health and health economics: HYEs (healthy-years equivalent) versus QALYs (quality-adjusted live-year). J Health Econ 12: 325–339. doi: 10.1016/0167-6296(93)90015-7
- 34. Mehrez A, Gafni A (1989) Quality-adjusted life years, utility theory, and healthy-years equivalents. Med Decis Making 9: 142–149. doi: 10.1177/0272989X8900900209
- 35. Mehrez A, Gafni A (1991) The healthy-years equivalents: how to measure them using the standard gamble approach. 11: 140–146. doi: 10.1177/0272989x9101100212
- 36. Hutchings A, Durand MA, Grieve R, Harrison D, Rowan K, et al. (2009) Evaluation of modernisation of adult critical care services in England: time series and cost effectiveness analysis. BMJ 339: b4353. doi:10.1136/bmj.b4353.
- 37. Kim JJ, Goldie SJ (2009) Cost effectiveness analysis of including boys in a human papillomavirus vaccination programme in the United States. BMJ 339: b3884. doi:10.1136/bmj.b3884.
- 38. Khazeni N, Hutton DW, Garber AM, Hupert N, Owens DK (2009) Effectiveness and cost-effectiveness of vaccination against pandemic influenza (H1N1) 2009. Ann Intern Med 151: 829–839. doi: 10.7326/0003-4819-151-12-200912150-00157
- 39. Bell CM, Urbach DR, Ray JG, Bayoumi A, Rosen AB, et al. (2006) Bias in published cost effectiveness studies: systematic review. BMJ 332: 699–703. doi: 10.1136/bmj.38737.607558.80
- 40. Miners AH, Garau M, Fidan D, Fischer AJ (2005) Comparing estimates of cost effectiveness submitted to the National Institute for Clinical Excellence (NICE) by different organisations: retrospective study. BMJ 330: 65. doi: 10.1136/bmj.38285.482350.82
- 41. Polyzos NP, Valachis A, Mauri D, Ioannidis JPA (2011) Sponsorship bias in assumptions of cost-effectiveness analysis: the diagnostic accuracy of the Pap test. CMAJ. In press.
- 42. Hoch JS, Dewa CS (2007) Lessons from trial-based cost-effectiveness analyses of mental health interventions: why uncertainty about the outcome, estimate and willingness to pay matters. Pharmacoeconomics 25: 807–816. doi: 10.2165/00019053-200725100-00001
- 43. O'Brien BJ, Sculpher MJ (2000) Building uncertainty into cost-effectiveness rankings: portfolio risk-return tradeoffs and implications for decision rules. Med Care 38: 460–468. doi: 10.1097/00005650-200005000-00003
- 44. Owens DK, Qaseem A, Chou R, Shekelle P, Clinical Guidelines Committee of the American College of Physicians (2011) High-value, cost-conscious health care: concepts for clinicians to evaluate the benefits, harms, and costs of medical interventions. Ann Intern Med 154: 174–180. doi: 10.7326/0003-4819-154-3-201102010-00007