Citation: Lavis JN, Røttingen J-A, Bosch-Capblanch X, Atun R, El-Jardali F, et al. (2012) Guidance for Evidence-Informed Policies about Health Systems: Linking Guidance Development to Policy Development. PLoS Med 9(3): e1001186. doi:10.1371/journal.pmed.1001186
Published: March 13, 2012
Copyright: © 2012 Lavis et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: The development of the Handbook was supported by a grant to the WHO by the Rockefeller Foundation, which had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. The paper represents the views of the authors and neither WHO nor the Rockefeller Foundation.
Competing interests: The Swiss Tropical and Public Health Institute and the Norwegian Knowledge Centre for the Health Services received funds from the WHO for the contributions of XBC and SL to developing the Handbook to produce health systems guidance, and some of this work is reported in this article. All other authors declare no competing interests.
Abbreviations: EVIPNet, Evidence-Informed Policy Network; WHO, World Health Organization
Provenance: Not commissioned; externally peer reviewed.
- Contextual factors are extremely important in shaping decisions about health systems, and policy makers need to work through all the pros and cons of different options before adopting specific health systems guidance.
- A division of labour between global guidance developers, global policy developers, national guidance developers, and national policy developers is needed to support evidence-informed policy-making about health systems.
- A panel charged with developing health systems guidance at the global level could best add value by ensuring that its output can be used for policy development at the global and national level, and for guidance development at the national level.
- Rigorous health systems analyses and political systems analyses are needed at the global and national level to support guideline and policy development.
- Further research is needed into the division of labour in guideline development and policy development and on frameworks for supporting system and political analyses.
- This is the second paper in a three-part series in PLoS Medicine on health systems guidance.
Policies about health systems can have profound impacts on citizens, patients, health professionals, and managers. For physicians, for example, the impacts can include changing their scope of practice (a governance arrangement), how they are paid (a financial arrangement), where they provide care (a delivery arrangement), and how their practices are supported in providing the types of care that citizens and patients value (an implementation strategy). Contextual factors are extremely important in shaping decisions about health systems, and policy makers have to work through all the pros and cons of different options before developing new policies on health systems.
In this paper, which is the second of a three-part series on health systems guidance ,, by considering issues raised during meetings of the World Health Organization's (WHO) Task Force on Developing Health Systems Guidance (Box 1), we:
- Explore the links between health systems guidance development and policy development at global and national levels;
- Examine the range of factors that can influence policy development.
The first article in the series makes a case for developing guidance to inform decisions on health systems-level questions based on an analysis of strategic health sector documents, explores specific challenges in producing such guidance, and identifies options for addressing these challenges . The third paper focuses on assessing how much confidence can be placed on health systems research evidence in both guidance and policy development processes .
Box 1. The Task Force on Developing Health Systems Guidance
To improve how it responds to requests for guidance on health systems, WHO established the Task Force on Developing Health Systems Guidance in 2009. Briefly:
- WHO selected the 20 members of this Task Force to ensure diversity in terms of skills and experience in four broad domains—health policy and systems research (30% of the panel members), systematic reviews (55%), national deliberative processes (20%), and guidance development (40%)—and in terms of gender (25% female) and current base in a low- and middle-income country (30%).
- The Task Force provided input to the development of a Handbook for Developing Health Systems Guidance and to the identification of broader issues that warranted further dialogue and debate .
- As part of this process, the Task Force reviewed approaches to developing clinical guidelines and the instruments used as well as the broader literature on the four broad domains listed above, which were synthesized by the Handbook developers.
- The Task Force suggested ways in which some of the approaches and instruments used to develop clinical guidelines could be adapted for use in the development of health systems guidance and indicated where there were important differences between these two types of guidance.
The writing group for this paper further considered the issues raised in these discussions and produced a first draft of the manuscript for comment by the Task Force. This paper and the other two in the series , were finalised after several iterations of comments by the Task Force and external reviewers.
Here and in the other two papers in the series, we rely on a set of key definitions (Text S1). While the definitions of health systems and health systems interventions may be familiar to many, the definition of health systems guidance differs significantly from the definition of clinical guidelines , both in its focus on including options for consideration and in its focus on using guidance to assist decision-making in a range of settings. The importance of contextual factors in shaping decisions about health systems means that health systems guidance should include information about what is known about the pros and cons of different options, the factors that will likely influence decisions about the options in different settings, and the tools that can support local guidance or policy development processes. A policy brief can be used to provide background evidence to inform a policy dialogue among stakeholders ,, which can in turn result in the articulation of the preferred policy option(s).
Importantly, in our definitions of both health systems guidance and policy briefs (Text S1) and throughout this paper, we emphasize the importance of being systematic and of involving all stakeholders. The recent World Health Assembly resolution on guidance for health system policies gives much needed attention to the issue of health systems guidance ,. However, the resolution contains only two uses of the phrase “evidence-based” (one for the assessment of a country's health and health system problems, the other for responses to evolving problems) and one call for “involv[ing] all relevant stakeholders.” Moreover, the resolution and related materials are largely silent on the need to follow systematic processes for evidence synthesis and stakeholder engagement.
Links between Guidance Development and Policy Development
Economy of scale and efficiency considerations at the global level, and resource and capacity constraints at the national level, mean that a division of labour among global guidance developers, global policy developers, national guidance developers, and national policy developers is needed to support evidence-informed policy-making about health systems (Figure 1). All these groups would help to set priorities and provide feedback (double-headed arrows in Figure 1).
Figure 1. Potential links between guidance and policy development at global and national levels.doi:10.1371/journal.pmed.1001186.g001
The first group—a panel charged with developing guidance about health systems at the global level similar to the panels convened by WHO to address specific issues—could likely best add value by developing health systems guidance and by supporting its use in three types of processes:
- Policy development at the global level (where applicable);
- Guidance development at the national level;
- Policy development at the national level.
Policy Development at the Global Level
International organizations like WHO and the World Bank, global initiatives like the Global Alliance for Vaccines and Immunization and the Global Fund to Fight AIDS, Tuberculosis and Malaria, and many bilateral initiatives and some very large multinational nongovernmental organizations (NGOs) could draw on the global guidance to inform decisions within their respective remits. Such decisions may be about how the organization invests its money directly (e.g., bulk-purchasing arrangements) or about how it attaches conditionalities to funds invested in related areas (e.g., staffing in HIV clinics). For some of these organizations and initiatives, making policy decisions about health systems is a relatively new area that lies outside their normally perceived remit, but the impacts of their involvement in making policy decisions on health systems could be profound given their scale of funding.
Guidance Development at the National Level
A national Evidence-Informed Policy Network (EVIPNet, see Text S1) sponsored by government or a guidance panel appointed by government (or other groups provided their members have the skills and experiences for such work) could draw on the global guidance to develop a country-specific policy brief that contextualizes the health system problem, options for addressing the problem, key implementation considerations, and (possibly) monitoring and evaluation considerations. The policy brief could draw on context-specific data and research evidence in each of these domains and could take into consideration any global decisions that might be considered binding on the country. A recent example of such an effort is the policy briefs that were developed to scale up the widespread use of artemisinin-based combination therapy to treat malaria. In this instance, global guidance spurred (but did not fully support) the national guidance development processes that EVIPNets undertook in several African countries and regions .
An EVIPNet could also convene a policy dialogue that allows the data and research evidence contained in the policy brief to inform and be considered alongside the views, experiences, and tacit knowledge of those who will be involved in, or affected by, future decisions about the health system problem . The evidence brief and dialogue summary could then be used by the national government to inform a decision.
Global decisions that might be considered in national guidance development processes include the normative standards that WHO member states collectively endorse at the World Health Assembly and agree to abide by in their respective countries. However, worryingly, a review of the statements made by the World Bank and WHO on topics addressed by the World Health Assembly between 2000 and 2003 (including three broad health systems topic areas) revealed that these statements are rarely consistent with the direction and the nature of effect claims from systematic reviews, which suggests that there is significant potential for improvement in how these organizations use or report the use of synthesized research evidence in their policy-development processes .
Policy Development at the National Level
A national government could make decisions for its country by drawing on the health systems guidance produced at the global level and/or a policy brief produced at the national level (with or without the summary of a policy dialogue that was informed by the policy brief) and by keeping in mind any global decisions that it acknowledges as binding on itself. The national government may choose to convene one or more policy dialogues or use other stakeholder-engagement approaches to support its decision-making and to reach agreement among stakeholders, ideally informed by the policy brief and supported by relevant tools highlighted in the guidance produced at the global level.
Will This Proposed Division of Labour Work?
We suggest that this proposed division of labour should be explored retrospectively by looking at several cases where health systems guidance has been developed at the global level and examining how the process could have unfolded differently with a view to refining the process. The division of labour could also be examined prospectively by examining cases where guidance is being developed at the global level and EVIPNets (among others) are drawing on this guidance to develop national policy briefs and convene national policy dialogues. Additional insights could be derived by examining the lessons learned in the field of health technology assessment where a similar need for a division of labour has been documented, albeit mainly at the level of drugs, devices, and other technologies .
Factors That Can Influence Policy Development
The final stage in the process of health systems policy development is the preparation of a document (or documents) produced within a national government to support decision-making by that government. A range of names might be applied to the document(s), which is ideally informed by global guidance, a national policy brief, and other inputs, but for simplicity, we refer to this decision-support document as a policy proposal.
The health systems guidance (which emphasizes global evidence), policy brief (which emphasizes global and national research evidence), and policy proposal (which emphasizes the many considerations supporting a preferred course of action) all need to synthesize what is known and not known about the pros and cons of the options under consideration. They also all need to assess the factors that can influence the choice, implementation, and monitoring and evaluation of these options in different settings (or in the case of the policy brief and policy proposal, at least refer to the synthesis contained in the guidance and policy brief, respectively, when they exist). While varying in emphasis, each of these documents would ideally document:
- Key features of an assessment about how to address a health system problem;
- Key features of a health system (or health systems) that can influence decision-making about how to address a health system problem;
- Key features of a political system (or political systems) that can influence decision-making about how to address a health system problem.
Key Features of an Assessment about How to Address a Health System Problem
An assessment about how to address a health system problem requires working through the underlying problem, the appropriate options to address the problem, and implementation issues (see Table S1). Much of what is known and not known about each of these areas can be derived from available data and research evidence. Health systems guidance that is produced at the global level can present overall summaries of the answers to these questions and identify patterns in the variation in these answers across health systems and political systems. Synthesized research evidence about health systems is increasingly available, and initiatives to make it easier to find and use evidence can support guidance development at the global level (e.g., Health Systems Evidence at http://www.healthsystemsevidence.org, which is now available in Arabic, Chinese, English, French, Portuguese, Russian, and Spanish). A policy brief produced at the national level can supplement the data and research evidence contained in global health systems guidance with local data and research evidence in order to present as clearly as possible what is known about how to address a health system problem in a particular country.
Key Features of Health Systems That Can Influence Decision-Making about How to Address a Health System Problem
An assessment of the key features of health systems that can influence decision-making about how to address a health system problem involves working through existing governance, financial, and delivery arrangements to determine which arrangements might help or hinder any options being considered (Table 1). A new health systems intervention such as pay-for-performance might “fit” into one health system, but the same intervention or part of the health system might require significant adjustment or complementary interventions to fit into a system where performance data are not collected systematically. Moreover, any new health systems intervention may have unanticipated consequences for the existing health system arrangements in which it is introduced, for example, by removing incentives for some types of activity.
Table 1. Key features of a health system that can influence decision-making about how to address a health system problem.doi:10.1371/journal.pmed.1001186.t001
Some of what is known and not known about the key health system arrangements can be derived from available data and research evidence. Health systems guidance that is produced at the global level can present overall summaries of what is known about which arrangements are key for achieving the desired impacts in different health system contexts. A policy brief produced at the national level can enrich these summaries with local data and research evidence in order to present as clearly as possible what is known about how existing health system arrangements may influence the selection and implementation of options. In this way, policy-making about health systems can be informed by a good understanding of the system-level context for an option and the range of its potential desired and undesired system-wide effects, and any adaptation and re-design of the option (and potentially other health system arrangements) that is needed to optimize synergies among health system elements.
Key Features of Political Systems That Can Influence Decision-Making about How to Address a Health System Problem
An assessment of the key features of a political system that can influence decision-making about how to address a health system problem involves working through the institutions, interests, and ideas that currently drive decision-making, as well as the “external factors” that can open windows of opportunity to introduce change (Table 2). As above, globally produced health systems guidance can ideally present overall summaries about what is known about which features are key and identify patterns in the variation of these features across health systems and political systems. A policy brief produced at the national level can enrich the data and research evidence from health systems guidance with local data and research evidence. Moreover, a policy dialogue conducted at the national level can further enrich the available data and research evidence with local views, experiences, and tacit knowledge about how the political system really works and which system features are most important for the issue at hand.
Table 2. Key features of a political system that can influence decision-making about how to address a health system problem.doi:10.1371/journal.pmed.1001186.t002
How Do These Assessments Relate to One Another and to Other Approaches?
As we discuss in the next paper, existing approaches to grading the quality of recommendations about clinical options will likely require significant modification for use in a health system context . The GRADE approach , for example, focuses only on two of the system-level factors that we have described—“values and preferences” and “feasibility”—but there are many more factors that will influence the choice of options for addressing a health system problem in different settings (as well as the implementation and the monitoring and evaluation of the preferred option). Also, the GRADE approach is typically executed by experts, and, while it can inform and complement policy dialogues and other stakeholder-engagement processes, it cannot substitute for these processes in the assessment of system factors.
Our proposed division of labour links guidance development at the global level with policy development at the national level. Balancing a broad range of system and political considerations to come to a reasoned judgment about how to address a health system problem is arguably the purview of those who have been given the accountability to make decisions about health systems, whether democratically elected or appointed. Typically, these policy makers are located at the national level. Our examination of the range of factors that can influence policy development (and that can be flagged for consideration in health systems guidance) highlights the need for rigorous system and political analyses in policy briefs at the national level. These analyses can be supported at the global level by health systems guidance that presents overall summaries about what is known about which health and political system features help and hinder particular options. These analyses can also be aided by the frameworks presented in Tables S1, 1, and 2, by the SUPporting POlicy Relevant Trials (SUPPORT) tools —a set of tools that can be used by people involved in finding and using research evidence to support evidence-informed health policy-making—and by a range of other approaches such as the evidence synthesis and grading approaches presented in the other two papers in the series , and the decision trees, system modeling, and evaluation frameworks presented in the Handbook for Developing Health Systems Guidance . Those involved in policy development at the national level will need to, as representatives of member states of WHO, push for change at the global level if guidance development in the area of health systems is to support the type of context-sensitive policy development at the national level that we have proposed here.
Certainly, there is a fruitful research agenda ahead for those interested in studying the division of responsibilities across guidance panels at the global level, EVIPNets or guidance panels at the national level, and national (or sub-national) governments, as well as the conditions under which global organizations have a legitimate role in making decisions about health systems. A complementary research agenda could focus on assessing frameworks and approaches to supporting system and political analyses, particularly in the difficult (but not uncommon) situation where a health system intervention is actually a complex bundle of interventions that can interact in helpful and unhelpful ways. The results of such research could inform ongoing modifications to the division of labour that we propose in this paper to ensure that limited resources are used wisely and that both the best available research evidence and the contextual insights of key stakeholders informs guidance development and policy development.
Translation of the Summary Points into Spanish by Xavier Bosch-Capblanch
Translation of the Summary Points into French by Bruno Clary, William Lenoir, and Lise Beck
Translation of the Summary Points into Portuguese by Bruno Viana
Translation of the Summary Points into Arabic by Fadi El-Jardali
Key features of an assessment about how to address a health system problem
Definitions used in this paper
We acknowledge other members of the Task Force on Developing Health Systems Guidance, who include (with their affiliations at the time when the Task Force was initiated): Edgardo Abalos, Centro Rosarino de Estudios Perinatales (Argentina); Abdul Ghaffar, Alliance for Health Policy and Systems Research, World Health Organization (Switzerland); Timothy Evans, Assistant Director-General, Information, Evidence and Research, World Health Organization (Switzerland); Davina Ghersi, Department of Research Policy and Cooperation, Information, Evidence and Research, World Health Organization (Switzerland); Metin Gulmezoglu, UNDP/UNFPA/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction, World Health Organization (Switzerland); Regina Kulier, Department of Research Policy and Cooperation, Information, Evidence and Research, World Health Organization (Switzerland); Peter Tugwell, Institute of Population Health, Department of Epidemiology and Community Medicine, and Department of Medicine, University of Ottawa (Canada); Tikki Pang, Department of Research Policy and Cooperation, Information, Evidence and Research, World Health Organization (Switzerland); Ulysses Panisset, Department of Research Policy and Cooperation, Information, Evidence and Research, World Health Organization (Switzerland); Gunn Vist, Norwegian Knowledge Centre (Norway).
Wrote the first draft of the manuscript: JNL. Contributed to the writing of the manuscript: JNL J-AR XB-C RA FE-J LG SL SO PO-Z AH. ICMJE criteria for authorship read and met: JNL J-AR XB-C RA FE-J LG SL SO PO-Z AH. Agree with manuscript results and conclusions: JNL J-AR XB-C RA FE-J LG SL SO PO-Z AH. Conceived and designed the process: JNL J-AR XB-C RA FE-J LG SL SO PO-Z AH.
- 1. Bosch-Capblanch X, Lavis JN, Lewin S, Atun R, Røttingen JA, et al. (2012) Guidance for evidence-informed decisions about health systems: rationale for and challenges of guidance development. PLoS Med 9: e1001185. doi:10.1371/journal.pmed.1001185.
- 2. Lewin S, Bosch-Capblanch X, Oliver S, Akl EA, Vist GE, et al. (2012) Guidance for evidence-informed decisions about health systems: assessing how much confidence to place in health systems evidence. PLoS Med 9: e1001187. doi:10.1371/journal.pmed.1001187.
- 3. Schunemann HJ, Fretheim A, Oxman AD (2006) Improving the use of research evidence in guideline development: 1. guidelines for guidelines. Health Res Policy Syst 4: 13.
- 4. Lavis JN, Permanand G, Oxman AD, Lewin S, Fretheim A (2009) SUPPORT tools for evidence-informed health policymaking (STP) 13: preparing and using policy briefs to support evidence-informed policymaking. Health Res Policy Syst 7: Suppl 1S13.
- 5. Lavis JN, Boyko J, Oxman AD, Lewin S, Fretheim A (2009) SUPPORT tools for evidence-informed health policymaking (STP) 14: organising and using policy dialogues to support evidence-informed policymaking. Health Res Policy Syst 7: Suppl 1S14.
- 6. World Health Organization (2011) Health system strengthening: improving support to policy dialogue around national health policies, strategies and plans. Report by the Secretariat to the 64th World Health Assembly. A64/12. Geneva: World Health Organization. Available: http://apps.who.int/gb/ebwha/pdf_files/WHA64/A64_12-en.pdf. Accessed 3 February 2012.
- 7. World Health Assembly (2011) WHA resolution 64.8: strengthening national policy dialogue to build more robust health policies, strategies and plans. Geneva: World Health Organization. http://apps.who.int/gb/ebwha/pdf_files/WHA64/A64_R8-en.pdf. Accessed 3 February 2012.
- 8. Lavis JN, Panisset UB (2010) EVIPNet Africa's first series of policy briefs to support evidence-informed policymaking. International Journal of Technology Assessment in Health Care 26(2): 229–232. doi:10.1017/S0266462310000206.
- 9. Hoffman SJ, Lavis JN, Bennett S (2009) The use of research evidence in two international organizations' recommendations about health systems. Healthc Policy 5(1): 66–86.
- 10. Velasco-Garrido M, Kristensen FB, Nielsen CP, Busse R (2008) Health technology assessment and health policy-making in Europe: current status, challenges and potential. Copenhagen: World Health Organization.
- 11. Kingdon JW (2003) Agendas, alternatives, and public policies. Second edition. New York: Longman.
- 12. Bosch-Capblanch X and Project Team (2011) Handbook for developing health systems guidance: supporting informed judgements for health systems policies. Basel: Swiss Tropical and Public Health Institute. Available: http://www.swisstph.ch/?id=53; http://www.swisstph.ch/fileadmin/user_upload/Pdfs/SCIH/WHOHSG_Handbook_v04.pdf. Accessed 3 February 2012.
- 13. Schunemann HJ, Fretheim A, Oxman AD (2006) Improving the use of research evidence in guideline development: 9. grading evidence and recommendations. Health Res Policy Syst 4: 21.
- 14. Lavis JN, Oxman AD, Lewis S, Fretheim A (2009) SUPPORT tools for evidence-informed health policymaking (STP). Introduction. Health Res Policy Syst 7: Suppl 1I1.
- 15. World Health Organization (2008) Everybody's business: strengthening health systems to improve health outcomes: WHO's framework for action. Geneva: World Health Organization.
- 16. Hamid M, Bustamante-Manaog T, Dung TV, Akkhavong K, Fu H, et al. (2005) EVIPNet: translating the spirit of Mexico. Lancet 366(9499): 1758–1760.
- 17. Lavis JN, Lomas J, Hamid M, Sewankambo NK (2006) Assessing country-level efforts to link research to action. Bull World Health Organ 84(8): 620–628.
- 18. Lavis JN, Oxman AD, Lewin S, Freitheim A (2009) SUPPORT tools for evidence-informed health policymaking (STP) 4: using research evidence to clarify a problem. Health Res Policy Syst 7: Suppl 1S4.
- 19. Lavis JN, Wilson M, Oxman AD, Grimshaw J, Lewin S, Fretheim A (2009) SUPPORT tools for evidence-informed health policymaking (STP) 5: using research evidence to frame options to address a problem. Health Res Policy Syst 7: Suppl 1S5.
- 20. Fretheim A, Munabi-Babigumira S, Oxman AD, Lavis JN, Lewin S (2009) SUPPORT tools for evidence-informed policymaking in health 6: using research evidence to address how an option will be implemented. Health Res Policy Syst 7: Suppl 1S6.
- 21. Hanson K, Ranson MK, Oliveria-Cruz V, Mills A (2011) Expanding access to priority health interventions: a framework for understanding the constrains to scaling-up. Journal of International Development 15(1): 1–14.
- 22. Lavis JN, Ross SE, Hurley JE, Hohenadel JM, Stoddart GL, Woodward CA, et al. (2002) Examining the role of health services research in public policymaking. Milbank Q 80(1): 125–154.
- 23. Weatherford MS, Mayhew TB (1995) Tax policy and presidential leadership: ideas, interests, and the quality of advice. Studies in American Political Development 9(Fall): 287–330.
- 24. Hall PA (1996) Politics and markets in the industrialized nations: interests, institutions and ideas in comparative political economy. Cambridge (MA): Centre for European Studies, Harvard University.
- 25. Pierson P (1993) When effect becomes cause: policy feedback and political change. World Politics 45: 595–628.
- 26. Coleman WD, Skogstad GD (1990) Policy communities and public policy in Canada: a Structural Approach. Mississauga (Canada): Copp Clark Pitman Ltd.
- 27. Arnold RD (1990) The logic of congressional action. New Haven: Yale University Press.
- 28. Olson M (1965) The logic of collective action: public goods and the theory of groups. Cambridge (MA): Harvard University Press.