Can multiple choice questions assess application of knowledge in pharmaceutical science teaching?
Keywords:
Assessment, Application Of Knowledge, Multiple Choice Question, Higher Cognitive Thinking, Pharmaceutical ScienceAbstract
Introduction: Application of knowledge is the required attribute of graduates from pharmacy schools in Australia and worldwide.
Aim: This paper describes a study, which examines the use of multiple-choice questions (MCQs) to assess the application of knowledge in two Pharmacy courses.
Methodology: Seventy-three MCQs were ranked independently in a ‘blind’ manner as knowledge recall (K) or application of knowledge (A). The ranked MCQs were then included in the final exams of the Pharmaceutics and Medicinal Chemistry courses. The differences in the percentage of students who obtained a correct answer for each of the types of MCQ were then analysed using Student’s t-test.
Results: No differences were observed in the percentage of students who obtained a correct answer for all MCQs compared to the percentage of students who obtained a correct answer for A or K MCQs in the Medicinal Chemistry exam. For the Pharmaceutics exam, significant differences were observed between the percentage of students who obtained a correct answer for all MCQs compared to the percentage of students who obtained a correct answer for A MCQs (p=0.012).
Conclusion: This is a primary study in the use of MCQs to assess the application of knowledge, and its findings indicated that great disparity in allocation of A versus K MCQs often exists between academics, which could have a profound impact on the overall outcomes of such study.
References
Abozaida, H., Parkb, Y.S. & Tekian, A. (2017). Peer review improves psychometric characteristics of multiple-choice questions. Medical Teacher, 39, S50-S54
Bloom, B.S., Engelhart, M.D., Furst, E., Hill, W.H. & Krathwohl, D.R. (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. New York: David McKay Company
Case, S.M. & Swanson, D.B. (2002). Constructing Written Test Questions For the Basic and Clinical Sciences. 3rd ed. Philadelphia: National Board of Medical Examiners
Collins, R. (2014). Skills for the 21st Century: teaching higher-order thinking. Curriculum & Leadership Journal, 12, 1448-1743
Fischer, M.R., Herrmann, S. & Kopp, V. (2005). Answering multiple-choice questions in high-stakes medical examinations. Medical Education, 39(9), 890-894
Freiwald, T., Salimi, M., Khaljani, E. & Harendza, S. (2014). Pattern recognition as a concept for multiple- choice questions in a national licensing exam. BMC Medical Education, 14, 232
Haladyna, T.M. (2013). Developing and validating test items. New York: Routledge
Harris, B.H., Walsh, J.L., Tayyaba, S., Harris, D.A., Wilson, D.J., Smith, P.E. (2015). A novel student-led approach to multiple-choice question generation and online database creation, with targeted clinician input. Teaching & Learning in Medicine, 27(2), 182-188
Miller, G.E. (1990). The assessment of clinical skills/ competence/performance. Academic Medicine, 65(9), 63-67
Mislevy, R., Wilson, M., Ercikan, K. & Chudowsky, N. (2000). Psychometric Principles in Student Assessment. Language Testing, 17, 163-184
Palmer, E.J. & Devitt, P.G. (2007). Assessment of higher order cognitive skills in undergraduate education: modified essay or multiple-choice questions? BMC Medical Education, 7, 49
Pugha, D., De Champlain, A., Gierl, M., Lai, H. & Touchie, C. (2016). Using cognitive models to develop quality multiple-choice question. Medical Teacher, 38(8), 838-843
Stupans, I. (2006). Multiple-choice questions: Can they examine application of knowledge? Pharmacy Education, 6, 59-63
Touchie, L. (2010). Guidelines for the Development of Multiple-Choice Questions. Medical Council of Canada.
Vanderbilt, A., Feldman, M. & Wood, I.K. (2013). Medical education: a review of course exams. Medical Education Online, 1, 1-5
Zaidi, N.L.B., Grob, K.L., Monrad, S.M., Kurtz, J.B., Tai, A., Ahmed, A.Z., Gruppen, L.D. & Santen, S.A. (2017). Pushing Critical Thinking Skills With Multiple- Choice Questions: Does Bloom’s Taxonomy Work? Academic Medicine. doi: 10.1097/ACM.000000000000 2087
