Automatic Bug Fixing via Deliberate Problem Solving with Large Language Models

Developers dedicate a significant share of their activities to finding and fixing defects in their code. Automated program repair (APR) attempts to reduce this effort by a set of techniques for automatically fixing errors or vulnerabilities in software systems. Recent Large Language Models (LLMs) such as GPT-4 offer an effective alternative to existing APR methods, featuring out-of-the-box bug fixing performance comparable to even sophisticated deep learning approaches such as CoCoNut. In this work we propose a further extension to LLM-based program repair techniques by leveraging a recently introduced interactive prompting technique called Tree of Thoughts (ToT). Specifically, we ask a LLM to propose multiple hypotheses about the location of a bug, and based on the aggregated response we prompt for bug fixing suggestions. A preliminary evaluation shows that our approach is able to fix multiple complex bugs previously unsolved by GPT-4 even with prompt engineering. This result motivates further exploration of hybrid approaches which combine LLMs with suitable meta-strategies.