Automated quality control procedures for real-time ocean observatories initiative datasets

The Ocean Observatories Initiative (OOI) is a project funded by the National Science Foundation which provides over 100,000 data products. OOI Cyberinfrastructure takes a two-pronged approach to data quality control: system level and human-in-the-loop. With system level, the system runs datasets through a series of six algorithms: global range, local range, stuck value, gradient, trend, and spike test. The resulting QC flags are encoded as binary integers that must be decoded in order for users to utilize them. The Rutgers data team uses an iterative approach to quality control of OOI data. Utilizing a suite of programming tools including the script, analyze_nc_data.py, the data team populates a standalone QC database with data reports. This QC database, in turn, populates the uFrame production system with data annotations that propagate to end users.