Predictive validity of the personal qualities assessment for selection of medical students in Scotland

Jon Dowell, Mary Ann Lumsden, David Powis, Don Munro, Miles Bore, Boikanyo Makubate, Ben Kumwenda

Research output: Contribution to journalArticle

16 Citations (Scopus)

Abstract

Background:The Personal Qualities Assessment (PQA) was developed to enhance medical student selection by measuring a range of non-cognitive attributes in the applicants to medical school. Applicants to the five Scottish medical schools were invited to pilot the test in 2001 and 2002. Aims:To evaluate the predictive validity of PQA for selecting medical students. Methods:A longitudinal cohort study was conducted in which PQA scores were compared with senior year medical school performance. Results:Consent to access performance markers was obtained from 626 students (61.6% of 1017 entrants in 20022003). Linkable Foundation Year (4th) rankings were available for 411 (66%) students and objective structured clinical examination (OSCE) rankings for 335 (54%) of those consenting. Both samples were representative of the original cohort. No significant correlations were detected between separate elements of the PQA assessment and student performance. However, using the algorithm advocated by Powis et al. those defined as 'non-extreme' (<±1.5 SD from the cohort mean scores; SD, standard deviation) character types on the involved-detached and on the libertariancommunitarian moral orientation scales were ranked higher in OSCEs (average of 7.5% or 25 out of 335, p=0.049). Conclusions:This study was limited by high attrition and basic outcome markers which are insensitive to relevant non-cognitive characteristics. However, it is the largest currently available study of predictive validity for the PQA assessment. There was one finding of significance: that those students who were identified by PQA as 'not extreme' on the two personal characteristics scales performed better in an OSCE measure of professionalism. Futures studies are required since psychometric testing for both cognitive and non-cognitive attributes are increasingly used in admission process and these should include more and better measures of professionalism against which to correlate non-cognitive traits.

Original languageEnglish
JournalMedical Teacher
Volume33
Issue number9
DOIs
Publication statusPublished - Sep 1 2011

Fingerprint

medical student
applicant
ranking
student
school
performance
examination
OSCE
psychometrics

All Science Journal Classification (ASJC) codes

  • Education

Cite this

Dowell, Jon ; Lumsden, Mary Ann ; Powis, David ; Munro, Don ; Bore, Miles ; Makubate, Boikanyo ; Kumwenda, Ben. / Predictive validity of the personal qualities assessment for selection of medical students in Scotland. In: Medical Teacher. 2011 ; Vol. 33, No. 9.
@article{0aac080e421c447491663d0383b6931a,
title = "Predictive validity of the personal qualities assessment for selection of medical students in Scotland",
abstract = "Background:The Personal Qualities Assessment (PQA) was developed to enhance medical student selection by measuring a range of non-cognitive attributes in the applicants to medical school. Applicants to the five Scottish medical schools were invited to pilot the test in 2001 and 2002. Aims:To evaluate the predictive validity of PQA for selecting medical students. Methods:A longitudinal cohort study was conducted in which PQA scores were compared with senior year medical school performance. Results:Consent to access performance markers was obtained from 626 students (61.6{\%} of 1017 entrants in 20022003). Linkable Foundation Year (4th) rankings were available for 411 (66{\%}) students and objective structured clinical examination (OSCE) rankings for 335 (54{\%}) of those consenting. Both samples were representative of the original cohort. No significant correlations were detected between separate elements of the PQA assessment and student performance. However, using the algorithm advocated by Powis et al. those defined as 'non-extreme' (<±1.5 SD from the cohort mean scores; SD, standard deviation) character types on the involved-detached and on the libertariancommunitarian moral orientation scales were ranked higher in OSCEs (average of 7.5{\%} or 25 out of 335, p=0.049). Conclusions:This study was limited by high attrition and basic outcome markers which are insensitive to relevant non-cognitive characteristics. However, it is the largest currently available study of predictive validity for the PQA assessment. There was one finding of significance: that those students who were identified by PQA as 'not extreme' on the two personal characteristics scales performed better in an OSCE measure of professionalism. Futures studies are required since psychometric testing for both cognitive and non-cognitive attributes are increasingly used in admission process and these should include more and better measures of professionalism against which to correlate non-cognitive traits.",
author = "Jon Dowell and Lumsden, {Mary Ann} and David Powis and Don Munro and Miles Bore and Boikanyo Makubate and Ben Kumwenda",
year = "2011",
month = "9",
day = "1",
doi = "10.3109/0142159X.2011.599448",
language = "English",
volume = "33",
journal = "Medical Teacher",
issn = "0142-159X",
publisher = "Informa Healthcare",
number = "9",

}

Predictive validity of the personal qualities assessment for selection of medical students in Scotland. / Dowell, Jon; Lumsden, Mary Ann; Powis, David; Munro, Don; Bore, Miles; Makubate, Boikanyo; Kumwenda, Ben.

In: Medical Teacher, Vol. 33, No. 9, 01.09.2011.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Predictive validity of the personal qualities assessment for selection of medical students in Scotland

AU - Dowell, Jon

AU - Lumsden, Mary Ann

AU - Powis, David

AU - Munro, Don

AU - Bore, Miles

AU - Makubate, Boikanyo

AU - Kumwenda, Ben

PY - 2011/9/1

Y1 - 2011/9/1

N2 - Background:The Personal Qualities Assessment (PQA) was developed to enhance medical student selection by measuring a range of non-cognitive attributes in the applicants to medical school. Applicants to the five Scottish medical schools were invited to pilot the test in 2001 and 2002. Aims:To evaluate the predictive validity of PQA for selecting medical students. Methods:A longitudinal cohort study was conducted in which PQA scores were compared with senior year medical school performance. Results:Consent to access performance markers was obtained from 626 students (61.6% of 1017 entrants in 20022003). Linkable Foundation Year (4th) rankings were available for 411 (66%) students and objective structured clinical examination (OSCE) rankings for 335 (54%) of those consenting. Both samples were representative of the original cohort. No significant correlations were detected between separate elements of the PQA assessment and student performance. However, using the algorithm advocated by Powis et al. those defined as 'non-extreme' (<±1.5 SD from the cohort mean scores; SD, standard deviation) character types on the involved-detached and on the libertariancommunitarian moral orientation scales were ranked higher in OSCEs (average of 7.5% or 25 out of 335, p=0.049). Conclusions:This study was limited by high attrition and basic outcome markers which are insensitive to relevant non-cognitive characteristics. However, it is the largest currently available study of predictive validity for the PQA assessment. There was one finding of significance: that those students who were identified by PQA as 'not extreme' on the two personal characteristics scales performed better in an OSCE measure of professionalism. Futures studies are required since psychometric testing for both cognitive and non-cognitive attributes are increasingly used in admission process and these should include more and better measures of professionalism against which to correlate non-cognitive traits.

AB - Background:The Personal Qualities Assessment (PQA) was developed to enhance medical student selection by measuring a range of non-cognitive attributes in the applicants to medical school. Applicants to the five Scottish medical schools were invited to pilot the test in 2001 and 2002. Aims:To evaluate the predictive validity of PQA for selecting medical students. Methods:A longitudinal cohort study was conducted in which PQA scores were compared with senior year medical school performance. Results:Consent to access performance markers was obtained from 626 students (61.6% of 1017 entrants in 20022003). Linkable Foundation Year (4th) rankings were available for 411 (66%) students and objective structured clinical examination (OSCE) rankings for 335 (54%) of those consenting. Both samples were representative of the original cohort. No significant correlations were detected between separate elements of the PQA assessment and student performance. However, using the algorithm advocated by Powis et al. those defined as 'non-extreme' (<±1.5 SD from the cohort mean scores; SD, standard deviation) character types on the involved-detached and on the libertariancommunitarian moral orientation scales were ranked higher in OSCEs (average of 7.5% or 25 out of 335, p=0.049). Conclusions:This study was limited by high attrition and basic outcome markers which are insensitive to relevant non-cognitive characteristics. However, it is the largest currently available study of predictive validity for the PQA assessment. There was one finding of significance: that those students who were identified by PQA as 'not extreme' on the two personal characteristics scales performed better in an OSCE measure of professionalism. Futures studies are required since psychometric testing for both cognitive and non-cognitive attributes are increasingly used in admission process and these should include more and better measures of professionalism against which to correlate non-cognitive traits.

UR - http://www.scopus.com/inward/record.url?scp=80051967455&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=80051967455&partnerID=8YFLogxK

U2 - 10.3109/0142159X.2011.599448

DO - 10.3109/0142159X.2011.599448

M3 - Article

VL - 33

JO - Medical Teacher

JF - Medical Teacher

SN - 0142-159X

IS - 9

ER -