计算机科学
计算机化自适应测验
过度拟合
校准
机器学习
可靠性(半导体)
参数统计
非参数统计
人工智能
样品(材料)
样本量测定
认知
可扩展性
数据挖掘
贝叶斯概率
项目反应理论
质量(理念)
限制
自适应采样
认知测验
心理测量学
形成性评价
采样(信号处理)
质量保证
贝叶斯定理
统计假设检验
作者
Pablo Nájera,Miguel Ángel Sorrel,Chia-Yi Chiu,Francisco José Abad
标识
DOI:10.3102/10769986251366581
摘要
This study examines innovative procedures for cognitive diagnostic computerized adaptive testing (CD-CAT) in small-scale assessments. Traditional CD-CAT methods, based on parametric cognitive diagnostic models (CDMs), often struggle with small calibration samples, leading to overfitting and overestimated reliability. Nonparametric alternatives, while more robust in small-scale settings, lack reliability information, limiting classification certainty and variable-length adaptive testing. To address these challenges, we propose four CD-CAT procedures using the parsimonious restricted deterministic input, noisy “and” gate (R-DINA) model, a parametric CDM tailored for small samples. Two of these procedures use a calibration sample (R-GDI and R-NPS), while the other two are calibration-free methods (R-NPS ML and R-NPS BM ). Through a simulation study, where calibration sample size, number of attributes, and item quality were manipulated, we compare these methods to the conventional CD-CAT based on the DINA model. Results indicate that R-GDI and R-NPS consistently outperform the conventional CD-CAT in terms of more accurate posterior probability recovery, classification accuracy, and balanced item usage, although they administer a larger number of items. The calibration-free methods also perform satisfactorily but exhibit reliability overestimation with low-quality items. Overall, the proposed procedures offer practical solutions for formative assessments in educational contexts characterized by small sample sizes and time constraints. We provide recommendations for the use and scalability of these methods in real educational settings.
科研通智能强力驱动
Strongly Powered by AbleSci AI