Evaluating machine learning algorithms for prediction of treatment response for sleep disturbances in patients with schizophrenia: A post-hoc analysis from a randomized controlled trial

BACKGROUND: A post-hoc analysis was planned to create and compare machine learning algorithms to predict treatment responses to sleep disturbances in patients with schizophrenia. SUBJECTS AND METHODS: This post-hoc analysis was done on a randomized controlled trial (NCT03075657), studying the effect of add-on ramelteon on sleep and circadian rhythm disturbances in 120 patients with schizophrenia. We created models using random forest, k-nearest neighbors, extreme gradient boosting machine, R part Classification and regression trees and logistic regression algorithms. R language with mlbench, caret, MASS, rPART packages were used. Box plot and dot plot were plotted to visualize comparisons among the models. RESULTS: The logistic regression algorithm was found to be the best-fit model with a specificity of 0.93 and sensitivity of 0.45, and ROC 0.78. Predominant symptom domain (positive or negative), urinary melatonin and global PSQI score at baseline were the most important variables when plotted in terms of mean decrease accuracy. These variables contributed significantly to the final model in the logistic regression algorithm, and the accuracy of this algorithm was found to be 90% for prediction. CONCLUSIONS: Machine learning models are an emerging trend in clinical research and should be translated into clinical practice. The logistic regression model predicted responders with 90% accuracy.