Modeling and Simulation to Support Public Health Outbreaks and Decision Making: Lessons Learned and Future Work

Tasked with protecting and improving the health of the people in their communities, public health departments face daily challenges that become even more daunting in the event of disease outbreaks and other public health emergencies. Decision making in these stressful situations is critical, and the use of data analytics and Modeling and Simulation (M&S) are valuable tools to support public health. Here we will review how M&S has changed with respect to supporting public health over the past decades to identify lessons learned as we try to incrementally improve our methods and approaches for future public health emergencies. We have experienced two pandemics over the past 15 years, and we will continue to face future disease outbreaks and biological threats so we must prepare for them now.

In the fall of 2001, the American public learned of a biological threat – anthrax. Specifically, the spores of Bacillus anthracis were sent in letters through the U.S. Postal Service (USPS), causing 22 people to become infected with anthrax and 5 deaths [1]. In the aftermath, approximately 300 postal facilities and other buildings were tested for the presence of B. anthracis spores, and approximately 32,000 people were given antibiotic Post-Exposure Prophylaxis (PEP) for potential exposure in Florida, the District of Columbia, New Jersey, and New York City [2]. Although it may be impossible to estimate the total cost of remediation for the facilities with potential anthrax spore exposure, it is estimated that more than $200 million was spent for the USPS facilities alone [3]. As a further response to the anthrax letters, the Project Bio Shield Act of 2004 was signed into law to help improve the development of Medical Countermeasures (MCMs) to protect the American public against Chemical, Biological, Radiological, Or Nuclear (CBRN) attacks [4]. Dr. Baccam led the team to support the Department of Health and Human Services (HHS) and the Public Health Emergency Medical Countermeasures Enterprise (PHEMCE) plan and prepare for scenarios involving biological agents through the use of M&S. Working with the interagency members of PHEMCE.

We reviewed the literature and used consensus techniques to build models that captured how people might become exposed and infected, and how they might fare in the presence or absence of different MCMs. These models helped to answer the myriad of “what-if” questions that ranged from scenario-specific exposures to the medical consequences of the availability of space (e.g., intensive care units - ICUs), staff (e.g., doctors, nurses, respiratory therapists), supplies (e.g., therapeutics, ventilators, medical oxygen), and standards of care (e.g., crisis standards of care). This process of preparedness planning with PHEMCE was applied to all the Category A biological agents and some Category B agents. In one example, we performed a sensitivity analysis involving approximately 12,000 simulations that was completed in 12 hours. While this type of modeling speed is sufficient for preparedness planning, it is not appropriate for responding to a real-world outbreak. For some biological agents, like anthrax, distribution and dispensing of PEP antibiotics were planned and explored through M&S. In order to save lives, high-throughput dispensing of PEP would need to occur at hundreds of Points of Dispensing (PODs), and plans were developed to achieve the desired throughput.

At the very onset of the H1N1 pandemic of 2009, initial cases in Mexico [5] caught the attention of health officials in the U.S. due to the travel of American students in and out of Mexico during their spring break activities. IEM was asked to review some H1N1 modeling results from the Department of Defense (DOD) contractors. A few days later, IEM was asked to provide some modeling analyses for potential new cases. Fortunately, although the H1N1 pandemic was characterized by higher-than-normal transmission rates (in comparison to seasonal influenza, but lower than historical influenza pandemics) between humans it exhibited a lower level of clinical severity in comparison to other historical influenza pandemics. The response modeling for H1N1 was fundamentally different than the preparedness modeling that we had previously done to support PHEMCE. In our planning models, we defined all the modeling assumptions for the disease characteristics, from incubation times and infection rates to hospitalization rates and survival outcomes to investigate the expected epidemic curve of new cases. In the response modeling for H1N1, we had very little data on the disease characteristics but only had access to the epidemic curve of new daily cases.

Thus, our modeling had to work backward in an effort to uncover the disease characteristics that might match with the observed case data. During the 2009 H1N1 pandemic, there were several factors that made disease modeling challenging. There were no rapid tests to identify H1N1 infections, so samples had to be sent to local or regional laboratories for testing. There was known to be under-reporting of cases as well as hospitalizations. Scarcity of detailed incubation data made it difficult to estimate the infection rates for H1N1. Data, as a whole, was challenging to obtain beyond the national or state level, and accurate hospitalization data was nearly non-existent. Due to these challenges, analysis of the impacts of the H1N1 pandemic continue to this day. Not long after ringing in the new year of 2020, mysterious cases of “undiagnosed pneumonia” were being reported in China. The first confirmed case in the U.S. appeared in an individual returning from Wuhan, China on January 21, 2020. With approximately 44,000 cases in China and 13 cases in the U.S., the World Health Organization (WHO) named this coronavirus disease (COVID-19) on February 11, 2020 and declared it a worldwide pandemic a month later.

By March 2020, it was clear that COVID-19 was drastically different than the H1N1 pandemic of 2009. By this time, the full genome of the virus, SARS-CoV-2, was sequenced, and real-time RT-PCR assays had been developed. Researchers in South Korea adapted bioterrorism plans of using PODs [6] developed from our earlier analysis of biothreat response for their COVID-19 drivethrough testing, enabling them to maximize their testing throughput [7]. Most importantly for modelers, daily new confirmed cases of COVID-19 were posted online for different countries and even broken down by states in the U.S. as well as giving data at the county level. Access to these data through different websites (Johns Hopkins, World meter, Our World in Data) allowed modelers to use the confirmed cases to develop projections for future COVID-19 cases. This access to case data was unprecedented and allowed modelers to focus on how the pandemic was changing in different locations over time. At IEM, we published our first COVID-19 case projections on March 31, 2020. Using a patent-pending artificial intelligence (AI) algorithm, we provided 7-day projections to over 375 counties, all 50 states, the District of Columbia, and three territories on a daily basis.

Each day, we performed approximately 1.6 billion simulations to produce all the necessary COVID-19 projections, completing those simulations in under 20 minutes. One of the hallmarks of our modeling approach was the assumption that the effective reproductive number, Rt, can and does change rapidly over short time spans (days) over the course of a pandemic. This approach forced us to examine multiple timeframes throughout the pandemic, with each timeframe associated with different effective reproductive rates. By April 2020, the Centers for Disease Control and Prevention (CDC) launched the COVID-19 Forecast Hub where researchers could submit their projections. The Forecast Hub combined all the submitted projections to create an ensemble forecast, which typically outperformed most of the individual projections. IEM submitted our U.S. COVID-19 projections to the COVID-19 Forecast Hub [8] as well as country-level projections of over 30 countries to the European Forecast Hub [9]. While our COVID-19 projections to forecasting hubs were published to help inform people, IEM also focused on supporting local decision makers.

We worked with local health departments to customize our COVID-19 projections to meet their needs. Nearly all our COVID-19 projections were provided pro bono, but we also helped some local public health departments with more in-depth modeling questions and analyses. One such jurisdiction was Los Angeles County, which has approximately 10 million residents spread across approximately 90 cities, 140 neighborhoods, and 120 unincorporated areas. We modeled the growth of new COVID-19 cases in these cities, neighborhoods, and unincorporated areas to identify hotspots with the greatest increase in new cases. Furthermore, the hospitalized cases of COVID-19 in ICUs and non-ICU beds were modeled to help project future demands on the medical system within the county. COVID-19 vaccines developed by Moderna and Pfizer received Emergency Use Authorization (EUA) by the Food and Drug Administration (FDA) in November 2020. In support of Los Angeles County Public Health, we were asked to model the potential benefits of COVID-19 vaccination for their residents. Based on vaccine efficacy data and some modeling assumptions, we projected the cases and deaths averted in comparison to no vaccinations.

With the COVID-19 vaccines available to the general public in mid-December 2020, President-elect Biden proposed an ambitious plan to administer 100 million doses of the vaccine in his first 100 days in office, more than doubling the vaccination rate when the vaccines first became available. Leveraging our plans for dispensing anthrax PEP, we estimated the personnel needs to achieve the president’s vaccination goal [10]. The Biden administration later updated their goal to administer the vaccine to 200 million people in 100 days, and IEM was able to adjust our modeling to investigate the potential benefits of this faster vaccination objective.

WHO declared an end to the COVID-19 global Public Health Emergency on May 5, 2023, and the U.S. COVID-19 Public Health Emergency declaration ended on May 11, 2023. Many public health officials still have a feeling of unease when reflecting on the COVID-19 pandemic. There was high attrition for public health officials during the COVID-19 pandemic and after its official end. Thus, it is critical to capture the lessons learned from the pandemic, our strategies and responses, and to plan for future disease outbreaks before we lose valuable institutional knowledge from the people who lived through it. A recently published commentary proposed some recommendations to help improve public health emergency readiness for future disease outbreaks [11]. Situational awareness is critical in any response, and the first recommendation is the development of a single, standardized national bio surveillance system to help facilitate this shared and common awareness.

While the COVID-19 case information available on websites was a wonderful source of data, we must improve on this by having a federally developed, deployed, and maintained system that State, Local, Tribal, And Territorial (SLTT) public health agencies can access. The design of this system must be a collaborative effort between federal and SLTT stakeholders to ensure that the system meets everyone’s needs. This system would greatly benefit rural and resource-limited communities, including tribal and territorial jurisdictions. The development of standardized definitions of public health and medical risk is the second recommendation from the commentary. Federal funding is typically dependent on assessment of risk, with tools developed to measure the risks and how SLTT agencies are addressing those risks. Unfortunately, the definition of public health and medical risk are not standardized, and the tools meant to assess risk are often subjective and categorical, making it difficult to apply or interpret the tools or results. The final recommendation from the commentary is the development of standardized metrics for preparedness and response.

These metrics are needed to better understand how to measure preparedness and response capabilities. Tools and resources that can support SLTT public health agencies include databases for bio surveillance data such as case reports as well as hospitalization and mortality data; disease modeling forecasting tools, including epidemiological estimates for incubation times and basic reproductive values; medical countermeasure dashboards that can inform production, procurement, and shipment information. These three recommendations aim to improve on the often disjointed and inconsistency observed during the public health response to COVID-19. It is no small task to get federal and SLTT public health agencies together, but the development of a standardized bio surveillance system and standardized definitions of risk and metrics for preparedness and response will greatly improve the coordination and collaboration for future disease outbreaks. National organizations such as the Association of State and Territorial Health Officials (ASTHO), the National Association of County and City Health Officials (NACCHO), the Council of State and Territorial Epidemiologists (CSTE), and others must be engaged in this effort to coalesce the SLTT and federal public health community.

While these standardized databases and tools would greatly help improve SLTT public health agencies with their planning and response efforts, it is unrealistic to imagine that these assets will meet the needs of all agencies. In a similar way, the CDC COVID-19 Forecast Hub and the case projections provide great information to decision makers, but there are often many questions that those projections cannot answer. IEM’s direct support of Los Angeles County, for example, came from their need for help to address their very specific questions that existing tools could not. Public health decision makers want to use data and M&S to answer their questions and provide them with scientifically accurate and defensible answers. I encourage those researchers who developed COVID-19 projections during the pandemic as well as other modelers to reach out to SLTT public health agencies and build collaborations. As modelers, we can learn the true concerns and pain points of public health officials from them directly, and that interaction will help us improve the models and tools that we can develop to help them.

We appreciate all the wonderful and great work that Brad Suchoski contributed to our COVID-19 modeling effort.

None.

None.

No conflict of interest.

1. Jernigan DB, Raghunathan PL, Bell BP, Brechner R, Bresnitz EA, et al. (2002) Investigation of bioterrorism-related anthrax, United States, 2001: epidemiologic findings. Emerg Infect Dis 8(10): 1019-1028.
2. Centers for Disease Control and Prevention (2001) Update: Investigation of bioterrorism-related anthrax and adverse events from antimicrobial prophylaxis. MMWR Morb Mortal Wkly Rep 50(44): 973-976.
3. (2005) Reopening public facilities after a biological attack: A decision making framework. The National Research Council. Washington, DC, USA.
4. (2004) Project BioShield Act of 2004. Public Law pp. 108-276.
5. Hsieh YH, Ma S, Velasco Hernandez JX, Lee VJ, Lim WY (2011) Early outbreak of 2009 Influenza A (H1N1) in Mexico prior to identification of pH1N1 virus. PLos ONE 6(8): e23853.
6. Baccam P, Willauer D, Krometis J, Ma Y, Sen A, et al. (2011) Mass prophylaxis dispensing concerns: traffic and public access to PODs. Biosecur Bioterror 9(2): 139-151.
7. Cox D (2020) The Baltimore bioterrorism expert who inspired South Korea’s COVID-19 response. Time Magazine.
8. Sherratt K, Gruson H, Grah R, Johnson H, Niehus R, et al. (2023) Predictive performance of multi-model ensemble forecasts of COVD-19 across European nations. Elife 12: e81916.
9. Lopez VK, Cramer EY, Pagano R, Drake JM, Odea EB, et al. (2023) Challenges of COVID-19 case forecasting in the US.
10. Bollyky T, Nuzzo JB, Baccam P (2021) How to distribute 100 million vaccine doses in 100 days. New York Times.
11. Rubin E, Harvey C, Villatoro A, Dean B (2024) Next generation public health emergency readiness: standardized tools and a threat agnostic bio surveillance system. Health Secur 22(2): 140-145.