External Validation and Modification of Nationwide Inpatient Sample Subarachnoid Hemorrhage Severity Score

BACKGROUND: The Nationwide Inpatient Sample Subarachnoid Hemorrhage (SAH) Severity Score (NIS-SSS) was developed as a measure of SAH severity for use in administrative databases. The NIS-SSS consists of International Classification of Diseases Ninth Revision (ICD-9) diagnostic and procedure codes derived from the SAH inpatient course and has been validated against the Hunt-Hess score (HH). OBJECTIVE: To externally validate both the NIS-SSS and a modified version of the NIS-SSS (m-NIS-SSS) consisting of codes present only on admission, against the HH in a Canadian province-wide registry and administrative database of SAH patients. METHODS: A total of 1467 SAH patients admitted to Ontario stroke centers between 2003 and 2013 with recorded HH were included. The NIS-SSS and m-NIS-SSS were validated against the HH by testing correlation between the NIS-SSS/m-NIS-SSS and HH, comparing discriminative ability of the NIS-SSS/m-NIS-SSS vs HH for poor outcome by calculating area under the curve (AUC), and comparing calibration of the NIS-SSS, m-NIS-SSS, and HH by plotting predicted vs observed outcome. RESULTS: Correlation with HH was 0.417 (P ≤ .001) for NIS-SSS, and 0.403 (P ≤ .001) for m-NIS-SSS. AUC for prediction of poor outcome was 0.786 (0.764-0.808) for HH, 0.771 (0.748-0.793) for NIS-SSS, and 0.744 (0.721-0.767) for m-NIS-SSS. Calibration plots demonstrated that HH had the most accurate prediction of outcome, whereas the NIS-SSS and m-NIS-SSS did not accurately predict low risk of poor outcome. CONCLUSION: The NIS-SSS and m-NIS-SSS have good external validity, and therefore, may be suitable to approximate traditional clinical scores of disease severity in SAH research using administrative data.