San Francisco’s Central COVID Response
June 24, 2021 by Meredith Trimble
When the coronavirus pandemic began, government agencies across the globe faced increased and rapidly changing resident needs. In order to orchestrate an effective response, these agencies needed to quickly aggregate disparate (and often sensitive) data and surface this data securely throughout the organization.
Such was the case in San Francisco. While the city was known for its open data portal, it lacked similar infrastructure for easy internal data-sharing across departments. This became apparent at the onset of the pandemic. The mayor’s declaration of a state of emergency immediately centralized government operations as staff moved to the COVID Command Center (CCC) housed in the Moscone Convention Center. The goal was to coordinate a centralized response — a response requiring centralized data-sharing.
“A coordinated response required multiple departments to share data with the staff and decision-makers of the newly formed CCC,” noted Blake Valenta, DataSF’s analytics strategist. “Their ability to make well-informed, rapid decisions was critically dependent on accurate, accessible data. While infrastructure existed for sharing within a department, distributing data from multiple departments across a newly created entity proved challenging.”
COVID-19 Data and Metrics Hub
The solution came in the form of a new COVID-19 Data and Metrics hub. DataSF’s three-person staff stood up the internal platform in a matter of weeks to collect and distribute accurate datasets to approved COVID Command Center parties.
Thanks to the city’s existing Open Data efforts, Socrata’s tools were familiar to DataSF as well as many of the rest of the city staff involved in the COVID-19 response. Leveraging similar infrastructure allowed DataSF to focus their efforts on curating trusted and easy-to-use datasets, rather than expending efforts on permission management or training staff to use a novel interface. With a landing page and 60+ datasets that ultimately powered more than 30 dashboards, the site hosted datasets on staffing and workforce, cases and testing, medical supplies, public outreach, and housing.
“The COVID-19 portal provided the city a clear, demonstrated win behind the power of internal data exchange,” Valenta said. As he shared with city and county peers during a March working group, the success of the portal in a crisis environment came largely from agile iteration based on the realities on the ground and identifying where they could provide the most value. “We provided a trusted, secure platform for sharing de-identified data that complemented the Department of Public Health’s robust internal data-sharing efforts.”
By focusing on their expertise in data structure, data governance and data management, DataSF were able to aggregate trusted and reliable data quickly in the portal. This allowed staff working with the City Performance Unit of the Controller’s Office, the Department of Public Health, the Department of Homelessness and Supportive Housing, and other departments from across the city to rapidly develop dashboards and reports that drove major CCC decisions.
According to DataSF’s Analytics Engineer Helen McLendon, “In urgent and rapidly progressing circumstances, a good data structure is anyone that can change quickly. The COVID-19 Data and Metrics Hub provided a convenient staging ground for iteration on novel datasets. We could quickly surface dataset drafts and documentation to key stakeholders across departments and respond to requests for new data elements and definitions in a timely fashion as pandemic conditions and our city’s response continued to evolve.”
From Internal Data to Open Data
DataSF leveraged its well-established Socrata Open Data infrastructure in the response as well. The initial COVID-19 Data and Reports webpage used a story pages feature that helped CCC analysts without a web development background quickly create web pages to host key public dashboards, indicators, and supporting text. This provided important public monitoring of the city’s response to the pandemic.
The site speaks to key public indicators, vaccinations, cases, deaths, population characteristics, demographics, testing, and more. Each report page links to the underlying open dataset, with an easy-to-use API connection that allows journalists and other external parties to develop their own reports. “Having open datasets associated with all our public dashboards creates a wider ecosystem of public transparency that is core to DataSF’s mission,” explained Tania Jogesh, DataSF’s data scientist.
In the future, the city plans to more widely apply the internal data-sharing platform for improved analytics in other areas. “From the urgency of the public health crisis, agreements and partnerships were formed to exchange data in a new way using a trusted platform,” Valenta said. “The success of these partnerships provided a compelling use case to expand the use of the internal data-sharing platform outside of the public health crisis.”
San Francisco won a 2021 Tyler Excellence Award for these COVID-19 data portals and its innovative approach to secure data exchange. The city’s scalable framework for internal data-sharing and governed process for releasing a subset of that data as open data to the public are ones any jurisdiction can adopt.