StructureBoost: Efficient Gradient Boosting for Structured Categorical\n Variables

Gradient boosting methods based on Structured Categorical Decision Trees\n(SCDT) have been demonstrated to outperform numerical and one-hot-encodings on\nproblems where the categorical variable has a known underlying structure.\nHowever, the enumeration procedure in the SCDT is infeasible except for\ncategorical variables with low or moderate cardinality. We propose and\nimplement two methods to overcome the computational obstacles and efficiently\nperform Gradient Boosting on complex structured categorical variables. The\nresulting package, called StructureBoost, is shown to outperform established\npackages such as CatBoost and LightGBM on problems with categorical predictors\nthat contain sophisticated structure. Moreover, we demonstrate that\nStructureBoost can make accurate predictions on unseen categorical values due\nto its knowledge of the underlying structure.\n