Developing and Validating a Multimodal Dataset for Neonatal Pain Assessment to Improve AI Algorithms With Clinical Data

Background: Using Artificial Intelligence (AI) for neonatal pain assessment has great potential, but its effectiveness depends on accurate data labeling. Therefore, precise and reliable neonatal pain datasets are essential for managing neonatal pain. Purpose: To develop and validate a comprehensive multimodal dataset with accurately labeled clinical data, enhancing AI algorithms for neonatal pain assessment Methods: An assessment team randomly selected healthy neonates for assessment using the Neonatal Pain, Agitation, and Sedation Scale. During painful procedures, 2 cameras recorded neonates’ pain reactions on site. After 2 weeks, assessors labeled the processed pain data on the EasyDL platform in a single-anonymized setting. The pain scores from the 4 single-modal data types were compared to the total pain scores derived from multimodal data. The On-Site Neonatal Pain Assessment completed using paper quality scales is referred to as OS-NPA, while the modality-data neonatal pain labeling performed using labeling software is MD-NPL. Results: The intraclass correlation coefficient among the 4 single-modal groups ranged from 0.938 to 0.969. The overall pain intraclass correlation coefficient score was 0.99, with a Kappa statistic for pain grade agreement of 0.899. The goodness-of-fit for the linear regression models comparing the OS-NPA and MD-NPL for each assessor was greater than 0.96. Implications for Practice and Research: MD-NPL represents a productive alternative to OS-NPA for neonatal pain assessment, and the validity of the data labels within the Multimodality Dataset for Neonatal Acute Pain has been validating. These findings offer reliable validation for algorithms designed to assess neonatal pain.