Decision Tree Extraction for Clinical Decision Support System With If-Else Pseudocode and PlanSelect Strategy

Decision trees, as a structured representation of medical knowledge, are critical resources for building clinical decision support systems. Their structured decision pathways can be used for retrieval to enhance clinical decision making. Currently, mainstream methods mainly utilize large language models and in-context learning for decision tree extraction. However, these methods often face challenges in understanding the structure of decision trees and accurately extracting the complete content of tree nodes, leading to noise in the extracted trees and ultimately impacting their effectiveness in clinical decision support system. To this end, in this paper, we propose a novel decision tree extraction framework, including two stages. In the first stage, we propose to use the If-Else pseudocode to represent the decision tree structure and design specific constraints on format and content to guide the LLM in generating outputs. In the second stage, we introduce a novel node-filling strategy called PlanSelect to match the extracted triplets with sub-sentences in the generated pseudocode, including four reasoning steps: observation, plan, action, and answer. To evaluate the effectiveness of our proposed method, we construct an English decision tree extraction dataset (EMDT) and conduct extensive experiments on the built and public datasets. Experiments on the Text2DT and EMDT datasets demonstrate that our method outperforms the current state-of-the-art approaches, achieving improvements of 1.37% and 1.54% on the $ER$ metric (which is lower is better), respectively. Furthermore, we use the medical decision trees extracted using our framework to improve the model's performance on clinical decision making tasks, i.e., CMB-Clin and MedQA.