Data Pipeline Design
KAKTI × Security Force Monitor
In a experimental collaboration with SFM, we are testing new data engineering processes to automate and enhance their rigorous and well-documented research methodology.
This work not only builds on lessons learned from the CAPstat project, but applies some interesting constraints around data synchronization. Particularly, there is no primary database of record, but rather data may need to be synchronized between spreadsheets, data files, and a publicly shared database.