Department of Social Policy & Intervention
University of Oxford
Tracking Policy Responses to COVID-19:
Opportunities, Challenges and Solutions
Mary Daly, Bernhard Ebbinghaus,
Lukas Lehner, Marek Naczyk and Tim Vlandas
Oxford Supertracker Policy Brief
14 September 2020
https://supertracker.spi.ox.ac.uk/
The Oxford Supertracker project is led by Mary Daly, Bernhard Ebbinghaus, Lukas Lehner, Marek
Naczyk and Tim Vlandas from the Department of Social Policy and Intervention (DSPI) at the University
of Oxford. The section on surveys is edited by Elias Naumann at the University of Mannheim. Technical
support is provided by Maximilian Trenkmann. The project is funded by the Economic, Social, Cultural
& Environmental Impacts of COVID-19: Urgent Response Fund funded by the Higher Education
Innovation Fund (HEIF) and the Oxford ESRC Impact Acceleration Account (IAA).
Executive summary
Governments have responded to the Covid-19 pandemic by adopting a wide range of policy measures
with different effects on infection rates and deaths, but also varying socio-economic consequences. The
Oxford Supertracker at the Department of Social Policy and Intervention (DSPI) aims to provide a global
online directory of relevant policy trackers that have been developed to monitor policies and individual
preferences in respect to Covid-19 across countries.
Trackers vary widely in terms of policy fields, country coverage, types of authors and users. The
Supertracker provides an online directory that brings together data sources on (1) deaths and cases of
Covid-19, (2) policies to prevent the spread of Covid-19, (3) preferences and behavioural responses of
individuals, and (4) policies seeking to mitigate or compensate for the consequences of Covid-19.
The Supertracker enhances research and knowledge opportunities in several ways. First, it facilitates
triangulation of information from different sources and fosters multi-dimensional analyses of policy
responses. Second, it provides information about the time structure of available policy and evidence,
which in turn makes it possible to adopt quasi-experimental research designs studying pre/post
treatment effects, for example. Third, the diversity of trackers generates potential complementarities
between them, which could be leveraged in different ways, for instance to reduce duplication, increase
coordination between authors of trackers thereby creating a momentum and resources for data
standardization and analysis.
Combining different databases with each other and using their temporal dimension may also generate
some challenges. First, there are comparability and combinatorial issues since the trackers typically
differ in their conceptualization of key policies as well as their data structures, country and time
coverage. Second, there are coverage and validity issues since academic institutions, NGOs and think
tanks tend to produce trackers according to their own particular designs, perceived need and available
resources. Individual trackers are often less global in coverage, less comprehensive, and up-to-date in
data gathering. Third, there are challenges in terms of policy impact evaluation because there is a lack
of up-to-date individual or household panel surveys that could help measure various outcome of the
pandemic – and policy responses – on particular social risk groups. Datasets created to monitor policy
responses to Covid-19 are difficult to leverage in research designs that require before-after comparisons
since they do not build on established classifications.
Reflecting on the situation we make the following recommendations for possible further action by
policy tracker producers, the research community at large and the Supertracker team. First, information
resources need to be categorized more precisely and rigorously so as to identify overlaps and gaps.
Second, we recommend addressing information gaps (e.g. in terms of uneven country coverage) and
using synergies, for instance through the creation of a Supertracker forum. This would be led by an
informal working group and host webinars and undertake joint project initiatives. Third, the
Supertracker should aim to become a fully-fledged data interface/repository providing direct access to
the content of any dataset listed in the directory.
2
1. Covid-19 and the boom in policy tracking
Since the outbreak of the Covid-19 pandemic, the crisis-driven adoption and implementation of policy
measures by governments has made it challenging to keep track of major developments. The rapidity
and scope of the pandemic has created a pressing need for up-to-date data collection and comparative
analysis. In this context, researchers from international organizations, think tanks, non-governmental
organizations (NGOs) and universities around the globe have developed Covid-19 policy trackers.
These are information databases that document policy responses and their consequences in real time.
In addition, researchers have fielded new surveys to map population preferences across countries and
topics since the outbreak of the pandemic.
All these policy trackers (and surveys) have provided an immensely valuable public good for informing
the public, policy-makers and researchers on ongoing developments. These information sources
facilitate comparative social research and evidence-based policy-making. The recent boom in the
demand for policy trackers is evidenced by skyrocketing Google searches for the term “policy
tracker[s]” during 2020 as shown in Figure 1 below.
Figure 1: Google searches for policy trackers, worldwide
Source: Google Trends (02.09.2020). Note: worldwide web search for all categories between 01/01/2020 and
31/08/2020. The y-axis denotes an index between 0 and 100 of relative search popularity standardised over the
time period indicated. The shaded grey area indicates the crisis period.
To document and help organize the rapidly increasing supply of novel policy trackers and to address
information overload, we have created the Oxford Supertracker: a global directory for Covid-19 policy
trackers and surveys. The online tool allows users to search and identify relevant information
resources, such as datasets, surveys, and systematic collections, across policy fields and countries. The
Oxford Supertracker project at the Department of Social Policy and Intervention (DSPI), University of
Oxford, has been tracking and assembling the numerous policy trackers that have emerged in the wake
of Covid-19. It provides an easily searchable online directory of more than hundred data sources (126
policy trackers and 44 surveys as of 7 September 2020). Although all sources are focused on the
pandemic, the policy trackers included – as well as the surveys on public attitudes and individual
behaviour – vary significantly in their policy focus, country coverage, authors/producers, and user
groups.
3
Table 1: Regional Scope of Policy Trackers (31 August 2020, N=126)
World / regions / countries N %
World-wide 82 65.1%
(incl. 12 OECD+, 3 G20/G south) 15 11.9%
Anglo (22 US/1 UK/1 CAN/1 AUS) 25 19.8%
Europe (12 EU, 1 East) 13 10.3%
Africa 5 4.0%
Asia 1 0.8%
All policy trackers 126 100.0%
In addition: directory of surveys (7 September 2020) 44
Different policy foci: Nearly all policy fields – ranging from education, fiscal, health, monetary and
social policy to the regulation of media, civic freedoms and elections – are relevant for an analysis of
the Covid-19 pandemic and its impact on economies and societies across the globe. Some trackers
cover a wide range of policies (such as containment measures or fiscal policy), while others choose to
detail specific aspects (such as prison systems, violent conflicts or paid sick leave).
A first glance reveals the evolution of policy trackers over time. Initially, early efforts concentrated on
tracing the demographic and epidemiological profile of the Covid-19 pandemic, including crucial
information on the number of cases, testing capacity, hospitalization, and mortality. Non-
pharmaceutical interventions, such as lockdowns and the stepwise “reopening” of economies, were
covered by several policy trackers. Further monitoring efforts focused on policy responses to mitigate
the socio-economic impact of the pandemic and public interventions, ranging from macro-economic
stimulus interventions to employment and social policies, such as furlough or short-time work
schemes. As they have evolved, more specific indicators on the economic and social impact of the
pandemic – for instance, GDP forecasts or unemployment rates – are becoming available.
Variation in country coverage: The large majority (82 out of 126, or about 65%) of the policy trackers
included in the Supertracker (by 31 August 2020) have a global scope (see Table 1), though many of
them do not systematically cover all UN recognized countries. Some report policy developments for a
small subset of countries scattered around the globe – e.g. OECD or G20 countries. Several trackers
(about 15%) focus on particular world regions, most notably Africa, Asia and the European Union or
Europe more generally. In addition, the Supertracker includes 25 national trackers (about 20%) from
anglophone countries, namely Australia, Canada, the United Kingdom and most often the United
States (typically including data on US states). Note that there are other trackers at national level but
they are not included here given that they are not English-language sources.
A variety of tracker authors: The country coverage of different policy trackers correlates somewhat
with the type of organization that has authored them. International organizations tend to have the
widest country coverage with UN international agencies typically providing the most comprehensive,
world-wide databases whereas OECD and EU agencies limit their country coverage to member states
and associated countries. This wide scope is associated with the wider geographic mandate of
international organizations and also their capacity to build upon ongoing data efforts and to mobilize
data contributions by member states or local offices.
4
Table 2: Providers of Policy Trackers (31 August 2020, N=126) by Organizational Type
Providers of policy trackers N %
Academics/institutes 42 33.3%
International organizations 33 26.2%
- UN…, ILO 18 14.3%
- OECD (12), EU (5) 17 13.5%
NGOs 21 16.7%
Thinktanks 12 9.5%
Companies (internet, etc.) 13 10.3%
Public agencies (US, CAN) 3 2.4%
All policy trackers 126 100.0%
In addition: directory of surveys (7 September 2020) 44
Among the producers (see Table 2), only a quarter of all policy trackers included in the Supertracker
are international organizations (UN-agencies, OECD, EU-related, etc.). Instead, the largest share (more
than one-third) of policy trackers included in the Supertracker originate from academic institutions
(including networks of scientists). These are often more selective in country coverage, more specialised
in terms of policy field, and more oriented towards evidence-based indicators. The other providers,
NGOs, think tanks and a few (sub)national agencies, provide diverse range of trackers with highly
varying geographical scope. Finally, global companies leverage the rise of big data to produce trackers
on significant behavioral aspects (e.g. mobility trackers) or information usage (e.g. Google search
terms).
Figure 2: Example of policy tracker search
Note: Searchable terms for https://supertracker.spi.ox.ac.uk/policy-trackers/
Title / Policy Areas / Focus (topics) / Country Coverage (worldwide or countries) / Data Format / Authors (data
producer)
5
Different user groups: Policy trackers not only serve multiple purposes and cover different types of
countries and policy fields, but they also provide information to different user groups. First, they may
help policy-makers – governments, interest groups, think tanks, international organizations, etc. – to
monitor policy developments, to benchmark countries in policy content and timing, and to learn from
best or poor practice. The information included in policy trackers can also help the public to acquire
knowledge of the different policy approaches to Covid-19 and assess their governments’ policy
responses to the pandemic by comparing them with those of other countries. Finally, policy trackers
provide up-to-date data allowing the global research community to generate new knowledge on the
causes and consequences of policy measures adopted in the wake of the pandemic thereby potentially
helping improve the evidence base for policy interventions.
2. Opportunities and challenges
The great diversity of Covid-19-focused policy trackers opens up new opportunities and related
challenges for their users and authors.
Opportunities for tracker users: Policy trackers enable policymakers, the public and academic or non-
academic researchers to follow different governments’ policy responses in a variety of policy fields in
real time. For policy research, two characteristics of trackers included in the Supertracker project are
particularly useful. First, when placed together in the Supertracker the huge diversity and richness of
trackers allows researchers to combine different database sources for a particular country or set of
countries. This country-by-country compilation facilitates triangulation of information from different
sources. Moreover, it also allows for multi-dimensional analyses on the many different policy
dimensions that have been relevant in responding to the pandemic. Second, some databases provide
events-based listings of policy measures (or interventions) and other databases even provide time
series in daily, weekly or monthly format. The inclusion of this temporal dimension in the data
structure of those databases can allow much more fine-grained analyses, both of the specific context
in which government measures have been adopted and of the impact of these measures, for example
through quasi-experimental (diff-in-diff) research designs studying pre/post treatment effects.
Related challenges: Combining different databases with each other and using their time dimension
may nonetheless be quite challenging for researchers.
Comparability and capacity for combining evidence: Different datasets may not be easily combined
because they typically differ in their conceptualization of key policies as well as their data structures;
they may therefore not be comparable or only in a limited way. Moreover, trackers and datasets
typically vary in terms of data type which also makes them more difficult to combine. Some policy
trackers provide policy description in (qualitative) text-based information or coding of policies (e.g.
Blavatnik government response index); other datasets provide quantitative information through
indicators of inputs (expenditure, benefit generosity) or outcomes (unemployment rate, poverty rate,
GDP decline). As already mentioned, datasets differ in terms of country coverage and time structure:
with many datasets having a cross-sectional structure while those with a temporal dimension may
store data in daily, weekly or monthly format.
Coverage and validity: Given that academic institutions, together with some NGOs, think tanks and a
few (sub)national agencies, tend to have more limited organizational and financial resources, they
produce trackers that are often less global in coverage, less comprehensive, and less frequently
updated. There are also questions about the validity of data on governments’ policy decisions. Datasets
6
typically code policy decisions when they are enacted or announced (some may even code simple
announcements of change through Twitter), but it is not clear whether those decisions are
implemented or how. Hence, there may be gaps in knowledge about implementation. Ascertaining the
details of roll-out and implementation would require further time-consuming analysis but is a
consideration.
Policy impact evaluation. Analysts face multiple challenges in leveraging the time dimension for policy
evaluation purposes. Although many trackers include fine-grained information on the timing of
different governments’ policy responses to the pandemic, this information may not be easily used for
experimental designs because there is a lack of up-to-date individual or household panel surveys that
could help measure various outcome of the pandemic – and related policy responses – on particular
social risk groups (e.g. ethnic minorities, elderly, etc.). Datasets that were specifically created to
monitor the policy responses, without building on established classifications, cannot easily be of
service in research designs that require before-after comparisons. For instance, certain surveys started
monitoring preferences only after lockdown began, making impact difficult to assess. The Oxford
Supertracker lists some fast response surveys that provide up-to-date information on changes in
attitudes and behaviour of individuals, which are of relevance to understanding whether/how social
inequalities are reproduced or intensified by Covid-19, but these datasets may not be sufficiently
detailed for fine-grained evaluations of policies adopted during the crisis.
Opportunities for tracker authors: The potential challenges met by users in combining or triangulating
data from different trackers create some opportunities for tracker authors and the research
community. Indeed, the diversity of trackers also means that there is potential overlap – or, to put it
more positively, complementarities – between them, which could be taken advantage of in different
ways. Overlap sometimes makes for unnecessary duplication of data collection efforts. Greater
coordination between authors of trackers could help them work out a clearer division of labour in
order to avoid duplication and make databases more complementary. This could also free up resources
for other tasks such as data standardization, data analysis, but also additional data collection on either
currently neglected policy dimensions (e.g. current lack of data on sources of financing – i.e. how
countries pay for covid-19-related programmes) or on important policies that require more fine-
grained data (e.g. comparative data on different dimensions – eligibility, coverage, max./min./average
benefit – of Covid-19-related cash benefits).
Related challenges: While greater coordination between tracker authors (data providers) could bring
benefits, it also presents potential issues about its desirability and feasibility. First, too much
coordination leading to too rigid a division of labour may not be desirable. Social phenomena and
policies may be conceptualized and measured in different ways depending on theoretical or practical
needs by data users or providers. Maintaining some degree of diversity in databases tracking the same
phenomena is therefore desirable. Second, coordination at an advanced stage of data collection may
not be feasible. Authors of trackers have already invested significant resources – “sunk costs” – into
creating their own taxonomies and carrying out very time-consuming data collection. Coordination
might only further strain limited resources particularly as the increasingly visible socio-economic
consequences of the pandemic may lead different organizations to put more emphasis on data analysis
than on data collection.
3. Recommendations for future development
Based on our analysis of the current state of Covid-19-related policy trackers compiled in the Oxford
Supertracker, we draw some conclusions by way of recommendation for possible further action by
7
policy tracker producers, the research community at large, and our own Supertracker efforts. We
suggest three sets of recommendations.
Systemically collecting and categorizing information resources: During the four weeks following its
launch, the Supertracker has been viewed and used by thousands of researchers from over 120
countries around the globe. Those researchers have also been contributing to the growing number of
entries listed on the website. This suggests a need for the type of coordination and platforming that
the Supertracker provides. By providing a platform that is easy to access by the policy and research
community as well as the public, the Supertracker allows all types of data providers - not only main
international organizations with wide-ranging communication channels, but also smaller organizations
with limited resources – to make their databases more widely known. Above all, it allows both users
and producers to identify where data collection efforts are overlapping and where gaps exist. It will be
important to maintain and regularly update the Supertracker in order to include any new data
developments by policy trackers (and surveys) related to Covid-19. Further efforts should be made to
add categorical information in order to better map the content of the databases.
Addressing information gaps and using synergies: Our analysis of trackers’ uneven country coverage,
of the diverse organizational map of data producers and of the different foci of the databases provides
a starting point for understanding information gaps and the potential for complementarities. Gaps take
the form of missing data and missing co-ordination of data collection across data producers. Through
direct engagement with authors/producers of policy trackers, it should be possible to support concrete
actions to fill information gaps in existing policy trackers and create synergies between them. There
are real opportunities for leadership in identifying and helping to find resources to fill gaps. The
Supertracker could contribute towards this by forming a Forum, for instance by setting up an informal
working group, holding webinars, and launching joint project initiatives, to enable all those who are
engaged in data collection to come together and exchange knowledge. Feedback from producers
indicate a need for cross-tracker and cross-institution coordination and also a willingness to explore co-
operation with the Supertracker as a resource. This could be implemented gradually by focusing on
particular topics and gathering specific sets of databases from data producers.
From an information directory to data interface/repository: One suggestion for further development
of the Supertracker, dependent on securing further funding, is to increase user accessibility, in
particular by providing direct access to the content of any dataset listed in the directory. This would
require the original data providers to agree on specific dataset formats and structures that would allow
the Supertracker to pull the data into its interface. This would have the advantage of allowing end-
users to merge and combine several data sources automatically and to go to a single source rather
than multiple sources. It would also make it possible for the data to automatically be updated on the
Supertracker when the original data creators add new observations to their datasets. Whereas the
push for replicability has moved many datasets into the dataverse (e.g. Harvard dataverse), the
Supertracker could also function as a repository of datasets before they are used for publication. Such
a user-friendly interface and repository would, however, require substantial investment into public
goods service provision by funding agencies as well as the willingness of data providers to make their
datasets compatible for such a Supertracker data sharing interface.
