12the International Ottawa Conference on Clinical Competence edition:12 location:New York City date:20-24 May 2006
Self-assessment is considered an important competence for the professional physician. It is however scantily taught in medical school and studies of its reliability have given contradictory results. Most previous research, with novices and experts alike, has shown poor correlations between self-assessment and objective tests. Cognitive psychology and perception theories have suggested possible explanations.
To measure the correlation between an objective MCQ test and two types of self-assessment: macro-prediction and macro-post diction.
Participants: General practice trainees at UCL were required to participate.
In order to overcome content specificity whilst limiting testing time to three hours, we chose to restrict the testing to four domains, two general domains (i.e. uro-genital, chest ) and two more specific domains (i.e. dermatology and locomotor). Participants were informed on the day of the four domains tested. Each domain was briefly described. It was specified that problems were those presenting in general practice and that they were expected to respond as a GP would in his/her practice. The first self-assessment questionnaire was submitted before the MCQ test. It comprised four 7 point-Likert scales, i.e. one for each domain. The MCQ contained 159 extended-matching questions. Questions were in mixed order. After the MCQ, another self-assessment questionnaire was submitted, comprised of four 7-point Likert scales (again one for each domain) and an estimation of MCQ score. Background variables were obtained before the testing session and included: gender, age, training environment, patient-contacts, perceived effort spent on learning activities and perceived progress.
Pearson correlations were calculated between MCQ scores (global and sub-domains) and both macro-level self-assessment tests.
127 general practice trainees (of 168) took part.
The macro-prediction instrument gave non-significant correlations. The macro-post-diction correlations were significant for the chest and dermatology sub-domains but observed correlations were poor, 0.2 and 0.312 respectively.
Two types of self-assessment instruments were used in this study. Results on prediction and post-diction showed poor correlation with external assessment.
Our study confirms previous research on self-assessment showing it to be poorly correlated with external assessments.