Health System Comparisons Framework
Health System Comparisons Framework
of reform experiences across countries. Arriving          bringing together the collection and allocation of
at a robust template is not straightforward, but the      funds, and split the chapter on organization and
Observatorys experience suggests some elements           management to address planning and regulation
that can help.                                            separately, reecting shifts in emphasis at the time
                                                          in wider academic and policy thinking. In addi-
                                                          tion, a new chapter was added, on the assessment
Structure                                                 of the health system, again a response to the more
                                                          explicit way this issue was being addressed at the
The HiT template benets from a clear structure,          time. The 2010 template condensed organization,
based on a functional perspective of health sys-          governance, planning, and regulation into a single
tems. It works from the premise that all health           chapter again and revised and extended the sec-
systems perform a number of nonnormative core             tion on performance assessment as policy-makers
functions (Duran et al. 2012), including the orga-        became increasingly interested in understanding
nization, the governance, the nancing, the gen-          and contextualizing the evaluations of their health
eration of physical and human resources, and the          systems that they were being confronted with.
provision of health services. The rst HiT tem-
plate was developed in 1996. It was revised in
2007 and again in 2010, but all iterations have           Scope and Content
used the notion of core functions and have drawn
on the literature and prevailing debate to interpret      There were of course other changes to the tem-
what those functions are.                                 plate between iterations in terms of the detail
   All revisions have involved input from staff           addressed within the relatively stable overall
(editors) and national authors, based on their            structure. New questions and issues were added
work on the country proles, but they have also           because areas like mental health, child health ser-
included consultation with a wider group of users         vices, and palliative care (2007) or public health
and stakeholders (Observatory Partners, various           and intersectorality (2010) came to the policy fore
units of WHO and of the European Commissions             and as a wide group of experts and users were
health directorate, and, more recently, members of        consulted. The 2007 template was particularly
the HSPM network). These review stages have               heavily laden with new additions and contributed
helped strengthen the template and build some             to longer and more time-consuming HiTs. Cer-
consensus around its structure and approach.              tainly there was a marked growth in the length
   Table 1 shows the changes over time and the            of HiTs in successive iterations with Estonia, for
very marked structural consistency between ver-           example, growing from 67 pages in 2000, to
sions. This is in part because of a conscious deci-       137 pages in 2004, and 227 pages in 2008. This
sion to adapt rather than rethink the structure           was addressed to some extent in 2010 with a
completely so that HiT users can read backwards           tightening of the template (see Box 3) after
in time as well as across countries. It is also a         which the 2013 Estonia HiT dropped to
testament to the robustness of the rst iteration.        195 pages, and it is being revisited again in the
The adjustments reect on a wider rethinking on           20152016 update.
how different elements t into the whole and on
what seemed more or less important at particular
                                                             Box 3: The 2010 Template, Structure
times.
                                                             and Contents
   The initial template placed more emphasis on
                                                             1. Introduction: the broader context of the
the political, economic, and sociodemographic
                                                                health system, including economic and
context and on a countrys historical background,
                                                                political context, and population health
because of the proximity to transition for so many
eastern European countries. The 20042007 revi-                                                  (continued)
sion consolidated nancing in one chapter,
6                                                                                                 B. Rechel et al.
user-friendly for authors and editors and secondly        on what belongs in the various subsections. How-
to create easier to read HiTs.                            ever, despite its denitions and advice on how to
    Key changes from the perspective of authors           produce a HiT, it is not a tool that can ensure
have been clear signposting of sections or sets of        consistency and comparability on its own. This
questions that are essential and of those which         is because health systems are so complex and
are only discretionary and some reworking of            there are so many layers of information that
the glossary elements and examples that charac-           could be deemed relevant. The Observatory has
terized the 1996 template. The intention in ag-          therefore developed a range of practice over the
ging what is and what is not essential is to help         last 20 years that helps make the template into a
authors and editors to focus and keep HiTs short          framework that supports health system
and easier to read and update. The editorial team         comparisons.
also drew up word limits for chapters, although
these have not been included in the published
template yet; they are used with authors to agree         Data Sources
the length of HiTs. The changes in the way terms
are explained reect the fact that they are now           Data is of course a constant issue in seeking to
familiar to authors and readers alike.                    make comparisons, particularly across countries.
    Key changes that have been aimed at readers           The Observatory has chosen to supply quantita-
include the reorganization of several subsections         tive data in the form of a set of standard compar-
to increase accessibility and clarity and the intro-      ative tables and gures for each country, drawing
duction of summary paragraphs with key mes-               on the European Health for All Database
sages at the start of chapters, an abstract (of less      (HFA-DB) of the WHO Regional Ofce for
than one page), and an executive summary                  Europe, as well as the databases from the Organi-
(of three to ve pages). These pull out                   zation for Economic Co-operation and Develop-
(or signpost) ndings in a way that allows                ment (OECD), the Eurostat, and the World Bank.
policy-makers and their advisers quick access             All of these international databases rely largely on
and is in line with the Observatorys growing             ofcial gures provided and approved by national
understanding of knowledge brokering (Catallo             governments. These are not unproblematic. The
et al. 2014) and the testing of HiT Summaries           WHO Europe HFA database covers the 53 coun-
between 2002 and 2008.                                    tries of its European region and Eurostat the
    There is a further round of revision which            28 EU member states and the 4 members of the
started in 2015 and is now being piloted, which           European Free Trade Association, while OECD
will ne-tune the HiT template. It will signpost          Health Statistics covers the 34 OECD countries
still more explicitly how health systems are doing        (of which only 26 are in WHOs European region
by integrating more evaluative elements in the            and 22 in the EU). There are also differences in
broadly descriptive sections rather than keeping        denitions and data collection methods. However,
them all for a single, policy-focused, assessment         they have the merit of being consistently compiled
section.                                                  and rigorously checked. National statistics are
                                                          also used in the HiTs, although they may raise
                                                          methodological issues, as are national and
HiT Processes: Making Sure                                regional policy documents, and academic and
Frameworks Are Used Consistently                          gray literature, although these do not of course
and Comparably                                            have comparability built in. Data in HiTs is
                                                          discussed and assessed, and there is explicit atten-
The HiT template in its various iterations has            tion given to discrepancies between national and
guided the writing of country proles, providing          international sources.
a clear overall structure, as well as detailed notes
8                                                                                             B. Rechel et al.
Authors, Author Teams, and the Role                      comparative framework and to acknowledge
of (Contributing) Editors                                their contribution are demonstrably worthwhile.
achieve expectations on quality and objectivity           is both a way of motivating authors to deliver on
and fulll the aims of the series.                        time and a way of securing impact when they
                                                          do. The Observatory has successfully tied HiTs
                                                          and HiT launches to EU Presidencies (Denmark
Dissemination and Policy Relevance:                       2012, Lithuania 2013, Italy 2014, Luxembourg
Helping Frameworks Achieve Their                          2015), to moments of political change (Ukraine
Objectives                                                2015), and to major reform programs in countries
                                                          (Slovenia 2016).
The HiTs are designed to allow comparisons
across countries, but they are not intended purely
to feed into (academic) research and analysis. HiT        Visibility
audiences are often national policy-makers who
use the HiT to take stock of their own health             HiTs can only be used when potential users are
system and to reach a shared understanding of             aware of their existence. The Observatory has
what is happening which different sectors, minis-         developed a mix of dissemination approaches to
tries, and levels of the health service (primary,         encourage uptake. There are launch events, typi-
secondary, regional, local) can all subscribe             cally in the country and in collaboration with
to. They use HiTs in considering reforms, as the          national authors, partner institutions, and Minis-
basis for policy dialogue and to explore policy           tries of Health. These work particularly well if
options, and to set their own health systems per-        linked to a policy dialogue (a facilitated debate
formance in a European context. Other users are           about policy options for decision-makers) or a
foreign analysts or consultants trying to get a           major national or international conference (like
comprehensive understanding of a health system,           the Polish annual National Health Fund meeting
and researchers and students. HiTs are a single           or the Czech Presidency of the Visegrad Group) or
source of information and pull together different         a workshop or meeting held by other agencies
strands of analysis which otherwise can be sur-           (European Commission meeting on health reform
prisingly hard to nd in one place.                     in Ukraine).
                                                              All HiTs are available as open access online on
                                                          the Observatorys web site and there are
Timeliness                                                e-bulletins and tweets to draw attention to new
                                                          publications http://www.euro.who.int/en/about-
Any comparative evidence will have more impact            us/partners/observatory. A list of the latest avail-
if it is delivered when it is still current and if it   able HiTs for the various countries is shown in
can coincide with a window of opportunity for             Box 4.
reform. The Observatory tries to turn HiTs around
in the shortest possible time, although this is not
                                                             Box 4: Latest Available HiTs, September 2016
always easy. The Health Systems and Policy
                                                             Albania HiT (2002)
Monitor is, in part, a response to this and provides
                                                             Andorra HiT (2004)
a log of policy developments and reforms online
                                                             Armenia HiT (2013)
in between formal HiT updates. Other steps to
                                                             Australia HiT (2006)
ensure that material is not superseded by develop-
                                                             Austria HiT (2013)
ments before it is published include agreeing a
                                                             Azerbaijan HiT (2010)
schedule with authors in advance, efforts to keep
                                                             Belarus HiT (2013)
HiTs short and focused, and quick turnaround on
                                                             Belgium HiT (2010)
review stages, all of which must be underpinned
                                                             Bosnia and Herzegovina HiT (2002)
by strong project management on the part of the
HiT editor. Linking HiTs to an entry point where                                                 (continued)
they are likely to be considered by policy-makers
10                                                                                    B. Rechel et al.
The Value of a Template                                   manuscript is ready for review. While this may
                                                          be less of an issue in frameworks with a narrower
A template that follows a rational and defendable         coverage, plans should include sufcient oppor-
structure, establishes a common vocabulary with           tunities for authors and editors to exchange views.
clearly dened terms (supported by examples
when appropriate), and is mindful of the way
researchers from different disciplines and national       The Need to Build In Accessibility
traditions may understand it is an invaluable tool.       and Relevance
It needs to include clear and sensible explanations
on how to use it, be sufciently robust to accom-         Users need to be considered in designing the
modate change over time, and allow a certain              template, the processes to deliver the compari-
degree of exibility. It should also reect on            sons, and the way ndings are disseminated.
what the nal output is expected to be and who            Readable, well-structured, well-presented reports
will use it.                                              that allow users to move from one report to
                                                          another and nd comparable information easily
                                                          will increase uptake and impact. Abstracts, sum-
The Importance of Author and Editor                       maries, and key messages will all help different
Roles                                                     users access the things they need. An example of a
                                                          cover and an executive summary of a HiT are
Comparative work demands data collection and              shown in Fig. 1 and Box 5. Delivering timely
analysis in different settings and national exper-        (current) data and analysis is also important if
tise is key to this. Selecting authors with appro-        the evidence generated is to have an impact.
priate skills and credibility is therefore essential      Reports that are overly long and detailed can still
and is boosted by clear criteria, by using teams          be useful, but they may tend to be used by aca-
rather than single authors, and by building long-         demics rather than policy-makers. Furthermore,
term relationships, which is possible through a           those developing comparative frameworks need
network like the HSPM. Good authors must be               to have an explicit debate as to how best to bal-
complemented by equally skilful editors who can           ance the comprehensive against the manageable
support the authors and ensure consistency.               and the timely. A mix of approaches to dissemi-
Bringing editors and authors together to agree            nation should be considered, paying attention to
expectations around timing and quality can be             ease of access, free download from the Internet,
extremely effective, as is keeping editors in             and translation into other languages.
touch with each other.
    The experience of the Observatory suggests
                                                             Box 5: Executive Summary from Germany,
that it is useful to provide for two roles analogous
                                                             Health System Review, 2014
to national author and HiT editor, to have clear
                                                             The Federal Republic of Germany is in cen-
(academic) criteria for guiding the choice of
                                                             tral Europe, with 81.8 million inhabitants
author, to schedule an initial meeting between
                                                             (December 2011), making it by some dis-
the editor and author(s) to go through the template
                                                             tance the most populated country in the
and clarify expectations, and to agree a clear time-
                                                             European Union (EU). Berlin is the
table. In the case of the HiT template, there is
                                                             countrys capital and, with 3.5 million resi-
often discussion of how to tailor the HiT to
                                                             dents, Germanys largest city.
national circumstances (and specically of which
                                                                 In 2012 Germanys gross domestic
areas will be addressed in more detail and which
                                                             product (GDP) amounted to approximately
in less), but this may not apply to other compara-
                                                             32 554 per capita (one of the highest in
tive frameworks. The experience with HiTs also
suggests there needs to be allowance for numer-                                                  (continued)
ous drafts and iterations before the overall
12                                                                                     B. Rechel et al.
                                                                                        (continued)
A Framework for Health System Comparisons: the Health Systems in Transition (HiT) Series. . .                13
                                                         Conclusions
     Box 5: (continued)
     for the German health-care system  as risk         The HiT series is, at least in Europe, in many
     pools differ and different nancing, access,        respects a gold standard for comparing health
     and provision lead to inequalities.                 systems. It has a long and positive track record
        Source: Busse and Blmel 2014                    with HiTs for 56 European and OECD countries,
                                                         often in several editions, and a total of some
                                                         130 HiTs overall. It has made information on
                                                         health systems and policies publicly available in
                                                         a format that cannot be found elsewhere and
The Need to Signal Credibility                           supported comparative analysis across countries,
                                                         including analytic studies, more detailed country
If evidence is to be used, the reader needs to have      case studies, and explicitly comparative works,
condence in it. Using expert inputs and consul-         for example on countries emerging from the
tation in developing the template can support this.      Soviet Union (Rechel et al. 2014), the Baltic states
External review stages of the HiT are of course          (van Ginneken et al. 2012), the central Asian
also important and ideally will include academic         countries (Rechel et al. 2012), or the Nordic coun-
and practitioner perspectives. It is also crucial that   tries (Magnussen et al. 2009). HiTs are some of
any review by governments or other authorities           the most downloaded documents held on the
with a potential conict of interests is handled in      WHO web site and are used not just in Europe
such a way that it is not seen to compromise the         but beyond. They have served as a guide for the
integrity of the work. Professional presentation,        Asia Pacic Observatory on Health Systems and
launches and links to major events, as well as           Policies (which was mentored by the European
other efforts to publicize the materials may           Observatory and launched in 2011) which uses an
also enhance the reputation of the work. Those           adapted version of the template to produce coun-
developing comparative frameworks will also              try reviews for its region. The average impact
have to be clear about the sources of data they          factor of (European Observatory) HiTs, calculated
use, their quality, and the extent to which they are     internally using Thomson Reuters methodology,
compatible with each other.                              was 3.6 between 2012 and 2014, with a high of
                                                         4.26 in 2013 although this only captures citations
                                                         in journals listed on PubMed/Medline. Google
The Need to Build in a Review Process                    Scholar, which also recognizes the gray literature,
                                                         shows that some HiTs achieve several hundred
The experience of the Observatory has shown the          citations per edition.
value of a comprehensive review process for                  The Observatorys experience with HiTs has
developing templates for health system compari-          generated insights that others developing frame-
sons. While it is clear that consulting widely           works for health system comparison might use-
brings new perspectives and creates acceptance           fully draw on. It demonstrates the importance of a
for a model, it does run the risk of diluting the        user-friendly template that helps authors and edi-
frameworks focus. The Observatory has found             tors produce accessible, relevant, and credible
that making it clear in advance that there are           outputs with a focus on what is expected from
space constraints and giving those consulted             the comparisons and on who is going to use
some explanation of how or why their suggestions         them. However, it also suggests that no template
have been acted on (or not) lessens the pressure to      is perfect. There are different ways of categorizing
expand the framework indenitely and helps               and grouping key functions (of a health or any
those consulted see that their inputs are valued         other system) or of conceptualizing systems and
even if they are not always used.                        different levels of tackling and reporting
A Framework for Health System Comparisons: the Health Systems in Transition (HiT) Series. . .                17
evaluation. To some extent these are a matter of          accepted tools (OECD et al. 2011) that help, but
preference. There are also and always tradeoffs           in many there are no agreed standard denitions
between comprehensiveness and accessibility,              (with health professionals being a case in point).
completeness and timeliness, and inclusiveness            Other comparative projects will need both to draw
and readability. The current HiT template can be          on the latest available knowledge and frameworks
seen as a pragmatic trade-off based on almost             and to invest in methodological work as the
20 years of experience. How other teams chose             Observatory team has done, for example, with
to balance these will depend on the focus of their        the conceptual model (the three-dimensional
comparisons and the people who are to use                 cube) to explain coverage (Busse et al. 2007;
their work.                                               Busse and Schlette 2007). They will also need to
    The Observatory has also found ways of com-           tailor responses to data and evidence availability
bining (excellent) national authors with its own          in parts of Europe (particularly but by no means
technical editors. This is not always straightfor-        exclusively in central, eastern, and southeastern
ward as not all European countries have the same          Europe) and to hope that EC/OECD/WHO initia-
capacity in health system research and national           tives on data will ultimately ll the gaps. There
experts with strong analytical and English writing        will still and inevitably be differences in the infor-
skills can be hard to nd (Santoro et al. 2016) and       mation available in countries, in the issues which
may move on rapidly. Moreover, HiT and HSPM               are important to them, and in the interests and
authors are not normally remunerated but, at              strengths of authors. Those developing frame-
best, receive only small honoraria. The HiT             works for comparison will have to address these
series has addressed these challenges by identify-        tensions in light of their overarching objectives
ing and linking formally with leading institutions,       and in the knowledge that health systems are
cultivating long-term relationships with HiT              constantly evolving. They may also nd, as the
author teams, and, most recently, through its             Observatory has, that a comparative framework
HSPM network. This mix of approaches may                  simply cannot capture everything and that analy-
have helped build capacity in countries. It has           sis for more specialized issues may require sepa-
certainly developed the understanding and                 rate study.
research (and people) management skills of the                Despite the challenges, the Observatory would
editorial team. Other comparative initiatives with        hold that there is real value in a framework for
limited resources might also want to consider             health system comparison, particularly one that
what they can do in terms of sharing ownership            relates to a dened user need and which can
and recognition to create non-monetary incentives         be sustained over time. Much follows from know-
for national counterparts and to develop their            ing who will use a set of comparisons and why.
own team.                                                 Longevity allows a framework to evolve  to
    Comparability is and will remain a challenge,         improve, strengthen comparability, and build up
despite the standard template, tables, and gures,        successive levels of knowledge. Combining the
and is likely to be an issue for all other compara-       two means a framework can move beyond the
tive projects. This is somewhat obvious when it           descriptive to the truly evaluative so that it cap-
comes to quantitative data given the divergent            tures and assesses aspects of health system perfor-
geographic coverage of international databases            mance in ways that speak to policy-makers or the
and the differences in denitions and data collec-        research community or, ideally, both.
tion methods, not to mention the challenges at the
individual country level. While it is clear that
caution must be exercised when comparing quan-            References
titative data from different sources, it is also true,
if less obvious, that qualitative data and the            Busse R, Blmel M. Germany: health system review.
                                                             Health Syst Transit. 2014;16(2):1296.
descriptive elements of the HiTs raise issues of
comparability. In some areas there are broadly
18                                                                                                      B. Rechel et al.
Busse R, Schlette S, editors. Health policy developments     Mossialos E, Allin S, Figueras J. Health systems in transi-
   issue 7/8: focus on prevention, health and aging, and        tion: template for analysis. Copenhagen: WHO
   human resources. Gtersloh: Verlag Bertelsmann               Regional Ofce for Europe on behalf of the European
   Stiftung; 2007.                                              Observatory on Health Systems and Policies; 2007.
Busse R, Schreygg J, Gericke CA. Analyzing changes in       OECD, Eurostat, WHO. A system of health accounts.
   health nancing arrangements in high-income coun-            Paris: OECD Publishing; 2011. doi:10.1787/
   tries: a comprehensive framework approach, Health,           9789264116016-en.
   Nutrition and Population (HNP) discussion paper.          Rechel B, Thomson S, van Ginneken E. Health systems in
   Washington, DC: World Bank; 2007.                            transition: template for authors. Copenhagen: WHO
Catallo C, Lavis J, The BRIDGE study team. Knowledge            Regional Ofce for Europe on behalf of the European
   brokering in public health. In: Rechel B, McKee M,           Observatory on Health Systems and Policies; 2010.
   editors. Facets of public health in Europe. Maidenhead:   Rechel B, et al. Lessons from two decades of health reform
   Open University Press; 2014. p. 30116.                      in Central Asia. Health Policy Plan. 2012;27(4):2817.
Duran A, et al. Understanding health systems: scope, func-   Rechel B, Richardson E, McKee M, editors. Trends in
   tions and objectives. In: Figueras J, McKee M, editors.      health systems in the former Soviet countries. Copen-
   Health systems, health, wealth and societal well-being:      hagen: World Health Organization; 2014 (acting as the
   assessing the case for investing in health systems.          host organization for, and secretariat of, the European
   Maidenhead: Open University Press; 2012. p. 1936.           Observatory on Health Systems and Policies).
Figueras J, Tragakes E. Health care systems in transition:   Richards T. Europes knowledge broker. BMJ. 2009;339:
   production template and questionnaire. Copenhagen:           b3871.
   World Health Organization Regional Ofce for              Santoro A, Glonti K, Bertollini R, Ricciardi W, McKee
   Europe; 1996.                                                M. Mapping health research capacity in 17 countries of
Magnussen J, Vrangbak K, Saltman RB, editors. Nordic            the former Soviet Union and South Eastern Europe: an
   health care systems. Recent reforms and current policy       exploratory study. Eur J Pub Health. 2016;26:34954.
   challenges. Maidenhead: Open University Press; 2009.      van Ginneken E, et al. The Baltic States: building on
                                                                20 years of health reforms. BMJ. 2012;345:e7348.