Subgroup identification based on differential effect search—A recursive partitioning method for establishing response to treatment in patient subpopulations

We propose a novel recursive partitioning method for identifying subgroups of subjects with enhanced treatment effects based on a differential effect search algorithm. The idea is to build a collection of subgroups by recursively partitioning a database into two subgroups at each parent group, such that the treatment effect within one of the two subgroups is maximized compared with the other subgroup. The process of data splitting continues until a predefined stopping condition has been satisfied. The method is similar to ‘interaction tree’ approaches that allow incorporation of a treatment‐by‐split interaction in the splitting criterion. However, unlike other tree‐based methods, this method searches only within specific regions of the covariate space and generates multiple subgroups of potential interest. We develop this method and provide guidance on key topics of interest that include generating multiple promising subgroups using different splitting criteria, choosing optimal values of complexity parameters via cross‐validation, and addressing Type I error rate inflation inherent in data mining applications using a resampling‐based method. We evaluate the operating characteristics of the procedure using a simulation study and illustrate the method with a clinical trial example. Copyright © 2011 John Wiley & Sons, Ltd.