Automate Datasets with Socrata Gateway
November 19, 2019 by Jake Thomas
Managing data is time consuming.
It must be gathered, cleaned, transformed, and prepared before any insights can be gleaned.
Studies show data analysts spend 80% of their time preparing data, and only 20% conducting actual analysis. And, that’s after meeting the high technical bar for getting data from legacy source systems onto a modern data platform.
Using your data with 21st Century tools ought to be easy.
That’s why we’re pleased to announce the general availability of Socrata Gateway.
Socrata Gateway is a new framework that allows users to create and automate data directly in Socrata without needing to write code. This functionality is poised to save analysts, administrators, and data publishers hours of time by streamlining workflows.
Socrata Gateway can be used to create an individual dataset, but the power lies in the automation to keep data updated without additional effort.
Socrata Gateway enables:
- Easier data ingress
- Reduction in data preparation
- Schedule updates automatically
Pierce County, an early adopter of Socrata Gateway, automates more than 30 datasets with the new functionality.
Jian Chu, a data analyst at Pierce County, says her workload has shifted from spending about 40% of her time preparing data versus automating it today with Socrata Gateway. One example is a dataset on capital projects, which includes data from a variety of county departments.
"Before, I had to contact users, get the updated file, and compile it together," Chu says. "Now, users can update the files on their time and the data will automatically be pushed to Socrata. I don’t have to spend time coordinating all the users, and they don’t have to wait for the final data."
By freeing up time once spent preparing or waiting for data, the team can spend more time conducting deeper analysis and exploring new insights, she says.
"The county has emphasized being data-driven, and we appreciate Socrata for providing the platform for it," Chu says.
The back-end platform connects directly to your on-premise and cloud-based data sources. This creates a single end-to-end workflow using the Socrata platform for the entire process — including data ingress, data preparation, transformation, and cleaning, as well as data publishing, and the ultimately scheduling updates.
Gateway allows you to connect through 40 plug-ins to on-premises databases, on-premise file directories, cloud-based sources, and Tyler systems, such as:
- SQL, Oracle, Access
- .CSV, Excel, and FTP
- Census, FBI
- NW Public Safety, Incode 10, New World ERP, and Infinite Visions
Pierce County makes use of the Excel plug-ins, on-premise file directories, and the U.S. Census data.
Gateway updates your datasets using the Socrata Dataset Management Experience, giving users the same tools they apply to edit, manage, transform, and schedule their data.
To get started setting up Socrata Gateway, click here or contact your Socrata account team.