PIE—Partially Interpretable Estimators with Refinement

We propose a new form of predictive models, Partially Interpretable Estimators (PIE), which jointly train an interpretable model and a black-box model to achieve partial model transparency while maintaining high predictive performance. Our design is motivated by prior research showing that interpretability does not require exposing all model details. Therefore, our objective is to explain the main components of the prediction, withholding complicated calculations that may not be necessary for users. PIE is designed to attribute a prediction to the contribution from individual features via a sparse linear additive model to achieve interpretability while complementing the prediction with a black-box model to boost the predictive performance. As such, the linear additive model captures the primary feature contributions, while the black-box component augments PIE’s predictive power by capturing the “nuances” of feature interactions as a refinement. Moreover, we include a sparsity constraint, allowing users to adjust the model to meet domain-specific needs of interpretability. To optimize predictive performance, we propose a coordinated training algorithm that jointly trains the two components of PIE. Experimental results show that PIE achieves accuracy comparable to state-of-the-art black-box models, with human assessments confirming that its interpretability is nearly equivalent to linear models. History: Accepted by Ram Ramesh, Data Science & Machine Learning. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2022.0098 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2022.0098 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .