Figures
Abstract
Background
While different ways of presenting treatment effects can affect health care decisions, little is known about which presentations best help people make decisions consistent with their own values. We compared six summary statistics for communicating coronary heart disease (CHD) risk reduction with statins: relative risk reduction and five absolute summary measures—absolute risk reduction, number needed to treat, event rates, tablets needed to take, and natural frequencies.
Methods and Findings
We conducted a randomized trial to determine which presentation resulted in choices most consistent with participants' values. We recruited adult volunteers who participated through an interactive Web site. Participants rated the relative importance of outcomes using visual analogue scales (VAS). We then randomized participants to one of the six summary statistics and asked them to choose whether to take statins based on this information. We calculated a relative importance score (RIS) by subtracting the VAS scores for the downsides of taking statins from the VAS score for CHD. We used logistic regression to determine the association between participants' RIS and their choice. 2,978 participants completed the study. Relative risk reduction resulted in a 21% higher probability of choosing to take statins over all values of RIS compared to the absolute summary statistics. This corresponds to a number needed to treat (NNT) of 5; i.e., for every five participants shown the relative risk reduction one additional participant chose to take statins, compared to the other summary statistics. There were no significant differences among the absolute summary statistics in the association between RIS and participants' decisions whether to take statins. Natural frequencies were best understood (86% reported they understood them well or very well), and participants were most satisfied with this information.
Conclusions
Presenting the benefits of taking statins as a relative risk reduction increases the likelihood of people accepting treatment compared to presenting absolute summary statistics, independent of the relative importance they attach to the consequences. Natural frequencies may be the most suitable summary statistic for presenting treatment effects, based on self-reported preference, understanding of and satisfaction with the information, and confidence in the decision.
Citation: Carling CLL, Kristoffersen DT, Montori VM, Herrin J, Schünemann HJ, Treweek S, et al. (2009) The Effect of Alternative Summary Statistics for Communicating Risk Reduction on Decisions about Taking Statins: A Randomized Trial. PLoS Med 6(8): e1000134. https://doi.org/10.1371/journal.pmed.1000134
Academic Editor: Glyn Elwyn, Cardiff University, United Kingdom
Received: July 17, 2007; Accepted: July 23, 2009; Published: August 25, 2009
Copyright: © 2009 Carling et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This study was funded by the Norwegian Research Council. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Abbreviations: ARR, absolute risk reduction; CHD, coronary heart disease; ER, event rate; NF, natural frequency; NNT, number needed to treat; RIS, relative importance score(s); RRR, relative risk reduction; TNT, tablets needed to treat; VAS, visual analogue scale(s)
Editors' Summary
Background
People often have to make decisions about their health care. Ideally, all the health care decisions that a person makes should be those best suited to his or her personal circumstances and expectations. Take, for example, someone with a high amount of cholesterol (a type of fat) in his or her blood. Because this condition increases the chances of developing potentially fatal coronary heart disease (CHD), such a person will often be advised by his or her doctor to take statins to reduce blood cholesterol levels. However, the person needs to consider both the benefits and downsides of this course of action. Can he or she afford to pay for statins, if their health care system requires him or her to? Does the person want to take a pill every day that might cause some side effects? That is, the person has to consider his or her “values”—the relative desirability of all the possible outcomes of taking statins—before deciding whether to follow his or her doctor's advice.
Why Was This Study Done?
It is well known that how information is presented to patients about treatment options and their consequences affects the choices that they make. For example, patients who are told that a drug will halve their chances of developing a disease (a 50% relative risk reduction) are more likely to decide to take that drug than those who are told it will reduce their absolute (actual) risk of developing the disease from 4% to 2%. Less is known, however, about which presentations of treatment effects best help people to make decisions that are consistent with their own values. In 2002, therefore, a series of internet-based randomized trials (studies in which participants are randomly allocated to different “treatment” groups) called the Health Information Project: Presentation Online (HIPPO) was initiated. Here, the researchers describe HIPPO 2, a trial that investigates how alternative summary statistics for communicating the reduction of CHD risk with statins affect people's decisions to take statins.
What Did the Researchers Do and Find?
Nearly 3,000 adults in Norway and North America rated the relative importance to them of CHD risk reduction, the cost of statins, and the need to take a daily pill through an interactive Web site. The researchers used these data to calculate a “relative importance score” (RIS), an indicator of each participant's values. Each participant then decided whether or not to take statins after being shown one of six summary statistics about the effect of statins on CHD risk (relative risk reduction and five indicators of absolute risk reduction). The presentation of the effect of statins as a relative risk reduction resulted in more people deciding to take statins over the whole RIS range than any of the absolute summary statistics. For every five participants shown the relative risk reduction statistic, an extra participant chose to take statins compared to the other summary statistics. When asked to compare the six summary statistics, the statistic that most people preferred and understood best was the “natural frequency,” an absolute summary statistic that gave the number of people likely to develop CHD with and without statin treatment.
What Do These Findings Mean?
Although these findings may not be generalizable to other populations or to other medical decisions, they provide new insights into how the presentation of information can affect the choices people make about health care. Specifically, these findings indicate that the presentation of the reduced risk of getting CHD as a result of taking stains as a relative amount is more likely to persuade people to take statins than several absolute summary statistics. They also suggest that the persuasive effect of the relative risk reduction summary statistic is not affected by the relative importance attached to the consequences of taking statins by individuals. That is, people shown the relative risk reduction statistic may be more likely to start statins to reduce their CHD risk (or a drug that reduces the risk of developing another disease) whatever their personal values than people shown absolute summary statistics. Finally, the findings on participant preferences suggest that natural frequencies may be the best summary statistic to include in tools designed to help people make decisions about their healthcare.
Additional Information
Please access these Web sites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.1000134.
- A PLoS Medicine Editorial discusses this trial and the results of another HIPPO trial that are presented in a separate PLoS Medicine Research Article by Carling and colleagues; details of a pilot HIPPO trial are also available
- The Foundation for Informed Medical Decision Making (a US-based non-profit organization) provides information on many aspects of medical decision making
- The Dartmouth-Hitchcock Medical Center provides information to help people make health care decisions through its Center for Shared Decision Making
- The Ottawa Hospital Research Institute provides also information on patient decision aids, including an inventory of decision aids available on the Web (in English and French)
- MedlinePlus provides links to information and advice about statins and about coronary heart disease (in English and Spanish)
Introduction
For patients, health care professionals, and policy makers to make informed choices about health care, they must have information about the effects of interventions that is valid and understandable. The manner in which this information is presented affects choice [1],[2]. The goal of the Health Information Project: Presentation Online (HIPPO) is to evaluate alternative ways of presenting research evidence in order to improve communication of information about the effects of health care and to facilitate clinical decisions that are consistent with patient values. (“Values” here refers to the relative importance of the desirable and undesirable effects of an intervention.)
Systematic reviews [1],[2] have found that use of relative risk reduction (RRR) to represent the effect of treatment results in individuals perceiving a larger treatment effect and being more likely to decide in favor of treatment compared with the use of absolute risk reduction (ARR) or the number needed to treat (NNT) [1]. In studies to find a minimally important difference, ARR produced 20% larger differences in the medians than NNT (25% versus 5%). The same review found that presenting the percentage of people experiencing outcomes with and without treatment resulted in more accurate perceptions than RRR when baseline risk was also provided. Others have advocated summary measures that have been less well studied, such as the tablets needed to take (TNT) [3] and “natural frequencies” (NF), i.e., raw observations that have not been transformed into percentages. Gigerenzer and others [4]–[7] advocate this format as a way of facilitating correct decisions, especially in diagnostics, among physicians as well as lay people. Hollnagel proposed a similar summary statistic, referred to as whole numbers [8].
There is a large literature that addresses how different presentations, including positive versus negative framing, different summary statistics, and different formats (numeric, verbal, or graphical) influence understanding, perceptions, and decisions; and how information about risk is used in decisions [1]–[16]. However, to our knowledge, there are no studies investigating the relationship between the summary statistic used and the extent to which decisions are consistent with individuals' values, apart from our pilot study [17].
Thus, we designed this study to assess the extent to which the use of different summary statistics affects choices about using statins to reduce the risk of coronary heart disease (CHD). We chose this decision because it is common, important, familiar to many, and because high-quality evidence is available about the benefits and downsides of statins [18]. It is a “preference sensitive” decision that is affected by patients' values [19]. Thus, among people with the same hypothetical or real risk of CHD, one would expect some degree of correlation between how important the desirable and undesirable consequences of taking statins are to them and the likelihood that they would decide to take statins. In other words, one would expect that people for whom the benefits of taking statins were less important and the downsides more important would be less likely, on average, to decide to take them than people for whom the benefits were more important and the downsides were less important.
Methods
Study Design
The CONSORT checklist and the protocol for this study are available as supporting information; see Text S1 and S2.
This study was an Internet-based randomized trial (Text S3) in which participants were randomized to one of six summary statistics presenting information about CHD risk reduction associated with statin use (Text S4). A pilot study of similar design with 770 participants in 2002 informed the final design of this trial [17]. Data from the pilot study are not included in this report.
Recruitment, Eligibility, and Allocation
The study and its Web site link were advertised to the public in Norway and North America through traditional media (radio, TV, flyers in public spaces including physician offices) and through online ads in Web portals and health-related Web sites. To encourage participation, we offered prospective participants the option to take part in a lottery for a $100 gift certificate and to receive study results.
Only complete responses from people who reported being 18 years or older, and that they were answering for the first time, were included in the analysis.
We randomized participants to one of the six presentations upon log-on by block-randomization (Text S4), using a looped sequence of 600 presentation assignments consisting of 100 blocks of six that was generated on http://www.randomization.com.
Data Collection
After choosing to log onto either the Norwegian or English-language version of the study Web site, participants received information about the study and gave informed consent to participate. Then, participants reviewed a hypothetical scenario involving themselves as patients with elevated cholesterol whose doctors had offered them the option of taking pills (statins) to lower blood cholesterol levels to reduce their risk of CHD (Figure 1). They learned that they had to take a pill each day and incur an out-of-pocket cost for the pills of US $50 (400 NOK) per month.
Then participants reported the relative importance they assigned to getting CHD, to a monthly increase of US $50 in health care costs, and to having to take a pill every day, using a horizontal 100-point visual analogue scale (VAS) with “No problem” and “Very difficult” as the lower and upper anchors.
Participants then considered the effectiveness of statins in reducing the risk of CHD using their randomly assigned summary statistic (Figure 2). The summary statistics reflected a 10-year CHD risk of 6% without statins (estimated risk for a person without other risk factors than a high cholesterol level [20]) and the 30% relative reduction in the risk of CHD with statins [21]. Participants then had to decide whether or not to take statins. They could access additional explanations of terms such as “angina” and “heart attack” using hyperlinks in the text (Figure 3) or navigate back to previously answered screens and change their answers, but could not return after having made a decision.
After making this decision, participants reported, using a 5-point scale where 5 was the highest rating, on their confidence in their decision and understanding of and satisfaction with the information. Participants then completed a numeracy assessment (Figure 4), a salience questionnaire (to measure how relevant or important the hypothetical scenario was likely to be to the participants) (Figure 5), and reported sociodemographic data. Then, they reviewed all six summary statistics and were asked which one they preferred.
Analysis
We calculated a relative importance score (RIS) for each participant by subtracting from her VAS score for CHD the sum of her VAS scores for increased health care costs and having to take a pill every day. We expected that higher RIS would be correlated with an increased likelihood of deciding to start taking statins.
The trial sought to test, in terms of concordance between decisions and RIS values, the following three hypotheses, based on the results of a pilot study [17]:
- RRR results in a higher likelihood of deciding to start taking statins across RIS values compared to the absolute summary statistics.
- The slope of the log odds of ARR is greater than the slope of the other absolute summary statistics.
- The concordance between decisions and values for the event rate (ER) is less than for the other absolute summary statistics; i.e. that the slope for the relationship between RIS values and the log odds of deciding to take statins is not significantly different from zero for ER (indicating that decisions were independent of the participants' elicited values), whereas it is positive (consistent with what would be predicted) and significantly different from zero for the other absolute summary statistics.
In order to evaluate the effects of the different summary statistics on the decision to start statins, taking into account each participant's RIS, we used the following logistic regression model:where D is the decision to go to take statins or not, G is the presentation group, S is the RIS value, and G*S is the interaction between the presentation and the RIS value. To make inferences about the response within each group and for the comparisons of groups we used dummy variable coding with reference parameterization for the presentation groups, i.e., directly estimating the difference in the effect of each nonreference level compared to the effect of the reference level. We used Wald tests to compare the p-values and confidence intervals (CIs) from the logistic regression and Chi-square tests to compare frequenices. The model was also explored by including numeracy and salience as covariates.
We used the model without covariates to test the three hypotheses based on the results of our pilot study [17] by (1) comparing the log odds of RRR to the log odds of the pooled absolute summary statistics for the RIS value at which the predicted log odds was zero (i.e. odds = 1) for the pooled summary statistics; (2) comparing the slope of the log odds of ARR to the slope of the pooled absolute summary statistics (excluding ARR), and (3) comparing the slope for the ER group to zero and testing for a difference in slope between ER and the slope of the pooled summary statistics excluding ER. Thus, we used a Bonferroni correction for four comparisons to adjust the confidence intervals for these analyses corresponding to an overall significance level of 0.05 (i.e. 0.05/4 = 0.0125) although the sample size estimates were based on three comparisons.
We performed additional comparisons for the difference in log odds at the median, 1st, and 3rd quartile values for RIS, and we compared the slope for the ER group to the slope for the pooled absolute summary statistics group (excluding ER) We did not adjust the 95% CIs or p-values for these or other comparisons. They should be interpreted with caution due to multiple testing since no assessment of their power was made in the protocol.
Sample Size
Using data from the pilot study, we estimated we would need about 750 to 800 participants per group to achieve 80% power to test each one of the three hypotheses at 0.0167 alpha level (after applying a Bonferroni correction).
Ethics
The protocol was reviewed and approved by the ethics committee of the University at Buffalo Medical School, Buffalo, New York. Participants gave informed consent via the Web site interface, having been given information about the study and told that they could quit at any time and request that their data be deleted. Contact information that some participants supplied in order to request the study results or to participate in the lottery was stored in a separate database that was not linkable with study data.
Results
The study recruited 2,978 eligible participants between June 2003 and July 2005 (Table 1). We decided to stop recruitment prior to achieving the intended sample size after multiple efforts to encourage participation over two years. We did not look at the results prior to deciding to stop the study. The six groups were similar, with respect to sex, age, years of education, salience of the scenario, and their elicited values. Most respondents were from USA (42%) and Norway (26%). Seventy-three percent of the participants chose the English language version of the Web site. Numeracy varied across the six groups from 65% that answered both questions correctly in the RRR group to 75% in the TNT and NF groups.
The most common ways in which participants reported finding out about the study were a link on another Web site (32%), an email invitation (27%), and a link to the study sent by a friend (18%). Fifty-nine percent of the participants were women, 54% were between 40 and 59 years old, and 63% had 17 or more years of education. Seventy-one percent answered both questions assessing numeracy correctly, and the scenario had low or no salience for 68%, based on their experience with CHD and hypercholesterolemia.
The participants' preferred presentation was natural frequencies (31%) closely followed by RRR (30%). Event rates were preferred by 20%, NNT by 10%, ARR by 5%, and TNT by 3%.
Decisions in Relation to Values
There was a significantly larger proportion in the RRR group that decided to start taking statins (74%) than in the “absolute” summary statistics groups (range 51% to 56%) (Table 2).
Participants in all groups were less likely to take statins when the relative importance score (RIS) was lower and more likely when it was higher (Figure 6). There were no differences in the association between RIS and the likelihood of taking statins across the five absolute summary statistics, including event rates (Table 3).
The testing of the three main hypotheses gave the following results:
- The difference in log odds of RRR versus the pooled absolute summary statistics at RIS = 12.3 (the point at which the predicted log odds was zero [odds = 1] for the pooled absolute statistics) was 0.88 (p<0.0001; odds ratio = 2.4; adjusted 95% CI 1.8 to 3.2). The odds of the RRR group choosing to take statins was two to three times greater at the median and the 1st and 3rd quartiles (Table 4) corresponding to differences of 20% to 21% in the proportions of participants choosing to take statins (Table 3).
- The difference in the slope of the estimated log odds of ARR to the slope of the pooled absolute summary statistics (excluding ARR) was 0 (p = 0.4, adjusted 95% CI −0.008 to 0.004).
- The slope of the log odds for the ER group was statistically significantly different from zero (β = 0.013, adjusted 95% CI 0.007 to 0.018) (Table 3), as it also was for the slope for the pooled absolute summary statistics group excluding ER (β = 0.0105, adjusted 95% CI 0.009 to 0.013). There was no significant difference between them: point estimate 0 (p = 0.3, adjusted 95% CI −0.004 to 0.008).
There were few responses for the salience categories of 0, 3, and 4, and few responders with no correct numeracy answers. We therefore pooled and renamed the categories of salience: “Low salience (0, 1),” “Some or high salience (2, 3, and 4)”; and numeracy: “Low numeracy (0, 1)” and “Correct (2)” and entered these covariates into the model. Salience was nonsignificant (p = 0.4). Although numeracy was significant (p = 0.01), it had a minor impact on the effect estimates and the standard error. The c-statistic for the overall model including covariates compared to the model without covariates was the same (c = 0.671).
Understanding and Satisfaction
In terms of understanding, 86% reported they understood NFs well or very well (Table 2). Confidence in the decision was not different across randomized groups (p = 0.62). Across all participants, confidence in the decision was associated with the decision to start statins (p<0.0001). Among those deciding to take statins 65% scored their confidence as 4 or 5 (anchored as “extremely confident” at a score of 5) compared to 51% among those who decided not to take statins.
The NF group was the most satisfied with the risk information that they had initially received (41% rated their satisfaction as 4 or 5 anchored as “extremely satisfied” at a score of 5); least satisfied were the TNT and the ARR groups (28% rated their satisfaction as 4 or 5) (Table 2). Across all participants, satisfaction in the risk information was associated with deciding to take statins (p<0.001).
Discussion
There was a clear relationship between participants' summary relative value scores expressed as RIS and the choices that they made across all summary statistics; i.e., as RIS increased, the likelihood of choosing to start statins increased, as would be expected for a preference sensitive decision. RRR resulted in the largest proportion of participants deciding to start statins compared to the absolute summary statistics. Participants in the RRR group were more likely to decide to start statins at all values of RIS than participants in the absolute summary statistic groups. The increased probability to choose to start statins in the RRR group is consistent with conclusions from other studies and supports the contention that RRR is a more persuasive summary statistic [1],[2],[13]–[16].
This is the first study of which we are aware that shows that people are more likely to be persuaded when presented with a relative summary statistic regardless of their values. The findings of this study confirm the results of our pilot study [17] with respect to RRR resulting in participants being more likely to decide to take statins regardless of their values. The findings did not support either of the two other hypotheses generated from the results of our pilot study. ARR did not result in a steeper slope of the estimate of the likelihood (log odds) of deciding to take statins relative to participants' RIS values. Event rates did not result in decisions disassociated from values and, in fact, produced an estimate that did not differ from the other absolute summary statistics.
In contrast to the pilot, where the majority (52%) preferred RRR, only 30% preferred RRR in this study and NF were preferred by 31% (compared to 25% in the pilot). In both studies, few participants preferred TNT (3% and 1% in this and the pilot study respectively) and ARR (5% and 4%). NNT was preferred by more participants in this study (10%) than in the pilot (4%).
Despite a 2-year recruitment period, the estimated sample size requirement was not met. This problem of recruiting participants to Internet-based studies has been reported elsewhere [22]. However, randomization worked well, generating six comparable groups.
Applicability of the Findings
We used a variety of strategies to recruit a convenience sample of participants, including placing links on other Web sites, sending email invitations, encouraging people to send the link to friends, and sending messages to discussion lists. Therefore, there is not a sampling frame from which participants were drawn to which we can compare them. The participants, however, had a relatively high level of education and it is uncertain that the findings are applicable to populations with less education [2],[17]. It is also uncertain to what extent results from the hypothetical scenario used in this study apply to actual decisions [13],[23], to personal communication [13], to other populations, or to decisions with higher risk levels.
While the results of Internet-based studies such as this one likely apply to printed as well as electronic information, they may not apply to personal communication, when it is possible to interact and adapt the presentation of information. The influence of how information is presented on decision making may also vary in relation to the salience of the scenario to decision makers [17],[24] and to their level of numeracy [17],[25], although neither significantly modified our results (data not shown).
Conclusions
Presentation of the RRR increases the likelihood of people accepting treatment over that of absolute summary statistics, independent of the relative importance they attach to the consequences. We did not find important differences in the relationship between decisions and values among the five absolute summary statistics we tested. However, natural frequencies may be preferable, based on self-reported preference, understanding and satisfaction with the information, and confidence in decision. This result supports the advice of others that natural frequencies should be the preferred summary statistic for decision aids and other risk communication tools [4]–[7].
Acknowledgments
We would like to express our deep appreciation to Jan Arve Dyrnes and Gro Alice Hamre for programming the Web pages that were used for this study and providing technical support; to Jon Helgeland and Jan Odgaard Jensen for statistical advice; and to Tom MacKenzie for helpful comments on an earlier version of this article.
Author Contributions
ICMJE criteria for authorship read and met: CLC DTK VMM JH HJS ST EAA ADO. Agree with the manuscript's results and conclusions: CLC DTK VMM JH HJS ST EAA ADO. Designed the experiments/the study: CLC DTK VMM JH ST EAA ADO. Contributed to the design of the study: HJS. Analyzed the data: CLC DTK. Collected data/did experiments for the study: CLC HJS. Wrote the first draft of the paper: CLC DTK. Contributed to the writing of the paper: CLC DTK VMM JH HJS ST EAA ADO.
References
- 1. McGettigan P, Sly K, O'Connell D, Hill S, Henry D (1999) The effects of information framing on the practices of physicians J Gen Intern Med 14: 633–642.
- 2. Moxey A, Dip G, O'Connell D, McGettigan P (2003) Describing treatment effects to patients: How they are expressed makes a difference. J Gen Intern Med 18: 948–995.
- 3. Skolbekken JA (1998) Communicating the risk reduction achieved by cholesterol reducing drugs. BMJ 316: 1956–1958.
- 4. Gigerenzer G (1996) The psychology of good judgement: Frequency formats and simple algorithms. Med Decis Making 16: 273–280.
- 5. Hoffrage U, Gigerenzer G (1998) Using natural frequencies to improve diagnostic inferences. Acad Med 73: 538–540.
- 6. Gigerenzer G, Edwards E (2003) Simple tools for understanding risks: from innumeracy to insight. BMJ 327: 741–744.
- 7.
O'Connor A, Llewellyn-Thomas H, Stacey D, editors. (2005) IPDAS Collaboration Background Document. February 17; 11. Available: http://ipdas.ohri.ca/IPDAS_Background.pdf. Accessed 18 July 2009.
- 8. Hollnagel H (1996) On the language of risk in the medical consultation [Danish]. Practicus 116: 237–239.
- 9.
Slovic P (2000) The perception of risk. London: Earthscan Publications.
- 10. Lloyd AJ (2001) The extent of patients' understanding of the risk of treatments. Quality in Health Care 10: i14–i18.
- 11. Ghosh AK, Ghosh K (2005) Translating evidence-based information into effective risk communication: current challenges and opportunities. J Lab Clin Med 145: 171–180.
- 12. Lipkus IM (2007) Numeric, verbal, and visual formats of conveying health risks: suggested best practices and future recommendations. Med Decis Making 27: 696–713.
- 13. Edwards A, Elwyn G, Covey J, Matthews E, Pill R (2001) Presenting risk information: a review of the effects of framing and other manipulations on patient outcomes. J Health Commun 6: 61–82.
- 14. Wills CE, Holmes-Rovner M (2003) Patient comprehension of information for shared treatment decision making: state of the art and future directions. Patient Educ Couns 50: 285–290.
- 15. Covey J (2007) A meta-analysis of the effects of presenting treatment benefits in different formats. Med Decis Making 27: 638–654.
- 16. Trevena LJ, Davey HM, Barratt A, Butow P, Caldwell P (2006) A systematic review on communicating with patients about evidence. J Eval Clin Pract 12: 13–23.
- 17. Carling C, Kristoffersen DT, Herrin J, Treweek S, Oxman AD, et al. (2008) How should the impact of different presentations of treatment effects on patient choice be evaluated? A pilot randomized trial. PLoS ONE 3: e3693.
- 18. Ebrahim S, Davey Smith G, McCabe C, Payne N, Pickin M, et al. (1999) What role for statins: a review and economic model. Health Technol Assess 3: 1–91.
- 19. O'Connor AM, Légaré F, Stacey D (2003) Risk communication in practice: the contribution of decision aids. BMJ 327: 736–740.
- 20. Anderson KM, Wilson PW, Odell PM, Kannel WB (1991) An updated coronary risk profile. A statement for health professionals. Circulation 83: 356–362.
- 21. LaRosa JC, He J, Vupputuri S (1999) Effect of Statins on Risk of Coronar Disease: A Meta-analysis of Randomized Controlled Trials. JAMA 282: 2340–2346.
- 22. Edwards A (2006) Presenting risk information to people with diabetes. Evaluating effects and preferences for different formats by a web-based randomised controlled trial. Patient Educ Couns 63: 336–349.
- 23. Wiseman D, Levin IP (1996) Comparing risky decision making under conditions of real and hypothetical consequences. Org Behavior Human Dec Proc 66: 241–250.
- 24. Tversky A, Kahneman D (1974) Judgement under uncertainty: Heuristics and biases. Science 185: 1109–86.
- 25. Schwartz LM, Woloshin S, Black WC, Welch G (1997) The role of numeracy in understanding the benefit of screening mammography. Ann Intern Med 127: 966–972.