Successes and lessons learned from OOI end-to-end system data quality audit

The Ocean Observatories Initiative (OOI) consists of seven research sites with over 800 instruments collecting ocean, seafloor, and meteorological data in the world's oceans, extending from the Irminger Sea to the Southern Ocean. The scale with which data are produced from the instruments and their platforms presents key challenges, including how to (1) build data and information management infrastructure that associates measurements from multiple instruments for concurrent observations, (2) develop data delivery mechanisms that meet a variety of needs, (3) ensure timely release of data, and (4) formulate capabilities to provide stable, longterm support for research and societal needs. As a strategy for maintenance of the sustained, large-scale, and variety of scientific observations collected, the OOI Cyberinfrastructure (CI) developed an end-to-end system designed to store, query, process, and disseminate the compiled information. These resources include raw instrument values and derived data products, metadata associated with instrument and platform deployments (e.g., deployment dates, water depths, instrument manufacturer, etc.), calibration coefficients, data provenance and descriptors of computational algorithms and transformation functions, and other related outputs. As with many data and information management systems, the need to monitor and improve upon the performance of the various interrelated components of the CI is an integral part in establishing the success of the system. For this reason, the data evaluation team at Rutgers initiated an end-to-end system data quality audit that ensures the system can accurately and completely deliver data within the required specifications. This was applied to a subset of representative platforms for all instrument types. Thus, based on specific system failures that can prevent production of quality data products, we prepared a set of tests and built tools that function as a troubleshooting method for system repair and enhancement. The method was used to report on the system's ability to: (a) Respond to data queries (b) Provide links to data product files (c) Produce all relevant data products (d) Produce correct provenance information (e) Produce quality science data products (f) Parse data with the correct specifications and the correct number of particles (g) Calculate different data levels with correct dimensions and units. To further assess the system fidelity to produce high quality data, a subject matter expert (SME) was consulted to provide outside validation of the OOI data quality, especially for some of the unique instrument classes that have less available documentation or publication records to consult. If the end-to-end system data quality audit logs successful system performance, the data checked were marked as ready to release to the public. If the data quality audit logs failures, they were investigated and a repair ensued, followed by another data check. Feedback from the end-to-end system data quality audit will be communicated to the public in form of annotations. This is needed to complete the end-toend system's ability to communicate information from and about the system. Considering that the data are continuously collected in the water and the system configuration can change over time, it is very important not to become complacent with the end-to-end system once it is in place. To ensure continuity of the required quality, an end-to-end system data quality audit is needed on regular basis.