单元测试
计算机科学
代码覆盖率
软件质量
稳健性测试
机器学习
背景(考古学)
软件工程
稳健性(进化)
软件
可靠性工程
人工智能
Python(编程语言)
测试用例
开发测试
软件容错
白盒测试
非回归检验
软件测试
集成测试
测试策略
故障覆盖率
数据挖掘
基于搜索的软件工程
故障检测与隔离
源代码行
测试驱动开发
断层模型
程序设计语言
作者
Jakub Walczak,Piotr Tomalak,Artur Laskowski
标识
DOI:10.1016/j.jss.2026.112834
摘要
Generative AI is gaining increasing attention in software engineering, where testing remains an indispensable reliability mechanism. According to the widely adopted testing pyramid, unit tests constitute the majority of test cases and are often schematic, requiring minimal domain expertise. Automatically generating such tests under the supervision of software engineers can significantly enhance productivity during the development phase of the software lifecycle. This exploratory study investigates the impact of code context and prompting strategies on the quality and adequacy of unit tests generated by various large language models (LLMs) from different providers, using 12 custom-developed Python methods designed specifically to avoid data leakage concerns. The results show that including docstrings notably improves code adequacy, while further extending context to the full implementation yields smaller incremental gains. The sequential multi-turn prompting strategy achieves the best results, with up to 96.3% branch coverage, a 57% average mutation score, and near-perfect compilation success rate. Among the six evaluated models, M5 (Gemini 2.5 Pro) demonstrated superior performance in both mutation score and branch coverage, while maintaining high compilation success rates. Notably, LLM-generated tests achieved comparable or superior mutation scores compared to tests written by a software practitioner, though with varying degrees of test redundancy. While limited in scope, this study provides initial insights into optimal configurations for LLM-based test generation. All custom code developed for this study and the resulting test suites are available at https://github.com/peetery/LLM-analysis .
科研通智能强力驱动
Strongly Powered by AbleSci AI