摘要
在认知诊断评估中,评价认知模型与作答数据的拟合非常重要。已有的层级相合性指标(HCI)仅能用于评价连接规则下模型与数据的拟合情况,有必要研究分离规则下相合性指标。HCI假设某项目上正确作答,便推断其子项目上的错误作答为失拟。由于作答反应的随机性,提出基于假设检验的项目相合性指标。该指标可用于区分连接规则和分离规则的作答数据、评价Q矩阵质量和衡量作答数据中的噪音、还可为评价认知模型和选择认知诊断模型提供参考。
From a psychological perspective, cognitive diagnostic models （CDMs） are divided into two basic categories called compensatory and noncompensatory diagnostic models depending on the interaction of skills. The interaction of skills in examinee’ item response behavior may sometimes be better modeled as disjunctive （e.g., psychological assessment） and other times be better modeled as conjunctive （e.g., mathematical assessment）. In other words, the choice of mode of attribute interaction clearly depends on the diagnostic setting. The selection of an appropriate CDM is based on analyses of the cognitive interaction between the skills and the items on the test. It almost always requires consultation of the literature and close collaboration among psychometric and substantive experts, in addition to empirical checking and confirmation. In conjunctive condensation rule, hierarchy consistency index （HCI） or item consistency index （ICI） can be directly used to assess whether actual examinees’ or items’ response patterns match the expected response patterns. It should be noted that the HCI or the ICI cannot be used with disjunctive CDMs where the mastery of all the attributes measured by an item is not necessary for successful performance because of the assumption that high ability on one attribute can compensate for low ability on other attributes. This leads us to propose the new indices that are specifically designed to identify misfits of item response vectors relative to disjunctive models. HCI or ICI requires the assumption that, from a correct or incorrect response, one can infer that the examinee has or has not mastered all attributes required by the item. Such an inference is often unreasonable and it is likely to draw the possibly incorrect inference. Considering statistical inference is generally more precise than everyday inference, we introduce a consistency index based on hypothesis testing to help detect misfitting item response vectors under the disjunctive condensation rule. The new consistency index with the original ICI can be used in the selection and evaluation of conjunctive or disjunctive model for data analyses. We also proposed a method to estimate slipping and guessing parameters based on comparison of item responses. To investigate whether the new indices can work well under certain conditions, simulated data are generated with an independent structure using five attributes. Four important factors were included in the design of the simulation study： （a） cognitive diagnostic model including two conjunctive models and a disjunctive model; （b） the quality of test Q-matrix with the four error rates from 0 to .4 with step .1, （c） the quality of items, and （d） the number of examinees with N = 500 or 1,000. The results show that these indices can be used to evaluate cognitive assumptions, to assess thequality of testing Q-matrix, to identify poor items with attribute misspecification and to estimate noise rate in the response data. The new consistency index with original indices may provide better understanding of the nature of cognitive assumptions and help determine which psychometric model is most appropriate and interpretable for the intended diagnostic assessment.
出处
《心理科学》
CSSCI
CSCD
北大核心
2015年第6期1496-1503,共8页
Journal of Psychological Science
基金
国家自然科学基金(31500909
31360237
31160203
30860084
31100756
31300862
61262080)
教育部人文社会科学研究青年基金项目(13YJC880060)
江西省社会科学研究"十二五"(2012年)规划项目(12JY07)
江西省教育科学2013年度一般课题(13YB032)
江西省教育厅科学技术研究项目(GJJ13207
GJJ13208
GJJ13209
GJJ13226
GJJ13227)
江西师范大学青年成长基金和江西师范大学博士启动基金的资助
关键词
认知诊断评估
分离缩合规则
连接缩合规则
项目相合性指标
Q矩阵质量
cognitive diagnostic assessment, disjunctive condensation rule, conjunctive condensation rule, item consistency index, Q matrix quality
