摘要
充分挖掘选择题(Multiple-Choice,MC)的诊断信息受到了较多关注,将干扰项信息考虑在内可以提升诊断精度。为了弥补参数模型基于大样本才能获得可靠估计的不足,以及适用于班级水平的小样本诊断测验情境,本研究提出了非参数的多选题诊断方法。模拟和实证研结果表明:(1)当MC测验中题目参数不存在较大差异时,d_(h-MC)法在多数情况下表现优于参数类诊断模型。(2)当MC测验中题目参数存在较大差异时,d_(ph-MC)法的表现最优。(3)实证研究中非参数方法和参数类模型的分类一致性程度较高,d_(ph-MC)距离法估计得到的考生属性总体掌握程度与总分相关最高。最后,基于MC诊断测验的特点提出了若干研究方向。
Cognitive diagnostic assessment(CDA)focuses on evaluating students'advantages and disadvantages in knowledge mastering,providing an opportunity for individualized teaching.Therefore,CDA has attracted attention of many scholars,teachers,and students at domestic and overseas.In CDA and a large number of standardized tests,multiple-choice(MC)are typical item types,which have the advantages of not being affected by subjective errors,improving test reliability,being easy to review,scoring quickly,and meeting the needs of content balance.To fulfil the potential of MC items for CDA,researchers proposed the MC-cognitive diagnosis models(MC-CDMs).However,these MC-CDMs pertain to parameter methods,which need a large sample size to obtain accurate parameter estimation.They are not suitable for small samples at class level,and the MCMC algorithm is very time-consuming.In this study,three nonparametric MC cognitive diagnosis methods based on hamming-distance are proposed,aiming at maximizing the diagnostic efficacy of MC items and being suitable for the diagnosis target of a small sample.Simulation study 1 considered four factors:sample size(30,50,100),test length(10,20,30),item quality(high and low),and the true model(MC-S-DINA1,MC-S-DINA2).Three nonparametric MC methods and two parametric models were compared.The results showed that in most conditions,the pattern accuracy rates and average attribute accuracy rates of the nonparametric MC method(d_(h-MC))were higher than those of parametric models,especially when the test length was short or item quality was low.In a real test situation,the quality of different items in a test may vary greatly.Based on this,simulation study 2 set the first half of the items at high quality and the remaining items at low quality.The results showed that the pattern accuracy rates and average attribute accuracy rates of the nonparametric MC method(d_(ph-MC))were higher than those of the parametric models in all conditions.In an empirical study,the nonparametric MC methods and the parametric models were used to analyze a set of real data simultaneously.The results showed that nonparametric MC methods and parametric models presented high classification consistency rates.Furthermore,the d_(ph-MC) method had satisfactory estimations.In sum,d_(h-MC) was suitable in most conditions,especially when the test length was short or the item quality was low When the quality of different items was quite diverse,d_(ph-MC) was a better choice compared with parameteric approaches.
作者
郭磊
周文杰
GUO Lei;ZHOU Wenjie(Faculty of Psychology,Southwest University,Chongqing 400715,China;Southwest University Branch,Collaborative Innovation Center of Assessment toward Basic Education Quality,Chongqing 400715,China)
出处
《心理学报》
CSSCI
CSCD
北大核心
2021年第9期1032-1043,共12页
Acta Psychologica Sinica
基金
国家自然科学基金青年项目(31900793)
北京师范大学中国基础教育质量监测协同创新中心重大成果培育性项目(2019-06-023-BZPK01)
中央高校基本科研业务费专项资金(SWU2109222)资助。
关键词
认知诊断评估
选择题
干扰项信息
非参数诊断方法
汉明距离
cognitive diagnostic assessment
multiple-choice item
distractor information
nonparametric diagnostic method
hamming distance
