The Efficacy of Rule-Based Versus Large Language Model-Based Chatbots in Alleviating Symptoms of Depression and Anxiety: Systematic Review and Meta-Analysis

BACKGROUND: The global mental health crisis is becoming increasingly severe. Due to the shortage of mental health professionals, high treatment costs, and insufficient accessibility of services, there is an urgent need for scalable and low-cost intervention methods. In recent years, chatbots have shown potential for psychological interventions. The efficacy differences between LLM-based and rule-based chatbots have not been systematically evaluated, with few studies directly comparing the two, and existing meta-analyses have notable limitations: there is high heterogeneity in intervention design (e.g., dialogue structure, interaction frequency, and duration) across studies, and there is a lack of direct comparison of differentiated intervention effects on depressive and anxiety symptoms, making it difficult to integrate conclusions. OBJECTIVE: By integrating studies from the past five years, this research evaluates the differences in effectiveness between LLM-based and rule-based chatbots in alleviating depressive and anxiety symptoms. It also analyzes the impacts of control group type, intervention duration, and age on intervention outcomes. By analyzing chatbot functionality, the study aims to provide evidence-based technological pathway options and optimization recommendations for differentiated interventions for depression and anxiety. METHODS: A systematic search of seven databases, included 15 studies published between 2020 and 2025. Robust variance estimation (RVE) was used to account for non-independent effect sizes, and standardized mean differences (SMDs) were calculated using Hedges' g. Based on the expectation of clinical and methodological heterogeneity among studies, a random-effects model was preselected, and the pooled effect size was estimated using REML and interpreted according to Cohen's criteria. Publication bias was assessed using the RVE-adjusted Egger test, funnel plot asymmetry, and a fail-safe N. RESULTS: For depression, rule-based intervention achieved a small but significant effect (g = 0.266, 95% CI [0.020, 0.512], p = 0.039), while LLM-based intervention showed a non-significant effect with wide confidence intervals (g = 0.407, 95% CI [-0.734, 1.550], p = 0.169). For anxiety, rule-based intervention did not yield a significant effect (g = 0.147, 95% CI [-0.073, 0.367], p = 0.152). Similarly, LLM-based intervention showed a higher point estimate but also with non-significance and wide confidence intervals (g = 0.711, 95% CI [-0.334, 1.760], p = 0.127). Subgroup analysis showed that Rule-based chatbot was more effective than the blank control for depression, with the greatest effect in the medium term (4-8 weeks). CONCLUSIONS: Rule-based chatbots have a modest effect on improving depressive symptoms and are suitable for environments with limited psychological resources. 4-8 weeks may be a critical intervention window. Intervention duration and participant age did not significantly influence intervention effectiveness. Limited by the sample size, robust evidence supporting the effectiveness of LLM-based chatbot interventions is lacking, and further sample size expansion is warranted. CLINICALTRIAL: Large Language Models; Chatbots; Mental Health; Depression; Anxiety.