PHSM stalkers have not been able to get their achievements easily. From developing data taxonomies to building organizational structures for data collection, cleaning and validation, PHSM the trackers launched their efforts without the benefit of precedent. At the start of the pandemic, trackers also worked without knowing each other’s efforts. While a big improvement over isolation, greater cooperation between trackers comes with its own set of challenges. Our review below provides a holistic look at the data and organizational challenges facing PHSM trackers individually and in groups. This can be completed by Shen et al.’s20 commentary, which provides a more in-depth discussion of the various data challenges faced by individual trackers.
Data taxonomy forms the basis for understandable and meaningful use of PHSM The data. Each tracker had different strategies for building their taxonomies, and given the particularities of how governments implemented COVID-19 PHSM, they usually developed them inductively and inferentially. The trackers have found that the main challenge is to develop a standard taxonomy that can both capture the nuances and particularities of a given country. PHSM deployment while allowing comparisons between countries. Additionally, ensuring taxonomies remain relevant over time by including periodic updates (e.g., documenting vaccination policies after global vaccine rollout) remains an ongoing challenge.
Similarly, data standardization remains a major challenge in PHSM data collection (as is the case for data collection more broadly). Beyond the enormous variability of definitions of policies and interventions PHSM trackers encountered while collecting data around the world, the lack of data standardization at national, state/provincial and local levels is a major barrier to data collection21. This issue not only affects COVID-19 data, but also basic demographic data. This is because detailed demographic data is often not publicly available, and definitions and categories of demographic characteristics vary by country and state.22. This mess not only makes data collection very difficult, but also makes it difficult to compare or identify the multitude of socio-economic and health consequences of the pandemic, especially with regard to the most vulnerable populations.
To collect, clean and validate this huge volume of PHSM data, most trackers rely on the tremendous contribution of volunteers. However, recruiting, training, engaging and organizing the corresponding volunteers present enormous challenges. Most volunteers are students and their availability therefore fluctuates according to the school calendar. The use of unpaid labor also raises issues of research ethics and sustainability. According to our survey, only about 10% of data collectors are paid; the vast majority are volunteers serving a public good (Fig. 3a).
Many trackers rely on volunteers for data collection not by design, but due to a lack of funding. Funding constraints are unfortunately quite severe: many political trackers have had to stop working due to lack of continued funding, resulting in large evidence gaps. When trackers receive funding, it’s often short-term due to uncertainty about how long the pandemic will last. According to our survey of trackers, only 16% of overall tracker funding needs are met (Fig. 3b). This resulted in a 65% drop in the number of trackers actively collecting data (Fig. 3c). Some trackers have attempted to address this issue by aligning their data in the few databases with more sustainable funding schemes, highlighting the importance of longer term funding for sustainable funding. PHSM data tracking.
PHSM trackers face challenges not only as individual players, but also as a collective ecosystem. At the start of the pandemic, 40+ PHSM follow the projects launched without knowing each other due to their emergency nature. These parallel data collection efforts have resulted in duplication of data, multiple taxonomy strategies between trackers, gaps in data coverage, and variations in data quality.23.
While there is significant data overlap between trackers, many trackers also have unique data coverage in specific areas, such as public health, economic policy, and human rights. While these differences provide a diversity of perspectives on PHSM tracking data, they can lead to difficulties in using the data. Working towards a single harmonized source may seem like an obvious solution, and indeed the World Health Organization (WHO) has done significant work towards this end.24. However, this work also highlights the difficulty of harmonizing data when the underlying data sources are still being cleaned and organized. More specifically, we think it is very important to continue to maintain diversity in project follow-up. This allows (i) different datasets to be validated against each other (ii) individual datasets to reflect a variety of research priorities and (iii) stakeholders to find the full set of data that best suits their needs.
The benefits of diversity must be constantly weighed against the costs of data collection, completeness and quality. With regard to the completeness of the data, PHSM the trackers have done an impressive job of documenting how governments around the world have responded to the pandemic at national and sub-national levels; however, overlaps and data gaps persist. In general, through PHSM trackers, data from the “Global North” is over-represented while data from the “Global South” is poor or missing. In the PHSM network, only one tracker has its core team physically based in the Global South. Due to donor interests, most data collection focuses on OECD countries and national policies, resulting in large gaps in data collection for less developed countries and sub-national levels . While more than 50% of the largest trackers collect subnational data (Fig. 3d), systematic subnational data collection for non-OECD countries is limited to Brazil, China, India, Russia and Nigeria.
Regarding data quality, trackers learned that local knowledge and/or language skills are essential to collect complete and accurate information. Because of this, PHSM Data quality for countries in the South is also more likely to suffer as many of the major trackers and their funders are based in the North.
Overall, while all trackers are united in their goal of documenting government responses to COVID-19, given the large number of policies it is possible to collect on one side, with the diversity of understandings of how to define a policy as well as organizational resources to capture them from the other side, there is wide variation in scope, quality and structure of PHSM data sets. While it is still premature to provide a definitive guide to which datasets are best suited for a given analysis, given the ongoing nature of the pandemic and accompanying data collection, Table 1 provides general guidance for deciding between different datasets with respect to temporal dimensions at the time of writing this commentary.
Ultimately, given the colossal volume and speed of government policy-making on COVID-19, greater collaboration between researchers from different fields (e.g., epidemiologists, political scientists, data scientists) as well as communication with policy makers is also needed to understand how best to model and analyze PHSM The data. Such work should start with better integration PHSM with other relevant COVID-19 data (e.g. COVID-19 cases, deaths and hospitalizations; economic indicators; environmental indicators). In all likelihood, further work would be needed to develop new analytical tools to use PHSM data to assess the drivers and impacts of the pandemic. While some trackers have made more progress than others on this front (for example, see Our World in Data’s COVID-19 Dashboard: https://ourworldindata.org/coronavirus; or the PERISCOPE COVID Atlas : https://periscopeproject.eu/covid-atlas), the field as a whole still lacks the necessary coordination and resources to move this work forward.
To address these challenges, in what follows, we outline key areas of intervention to PHSM data science and advocate for greater cooperation and communication among and betweenPHSM trackers.