An evaluation of 18 years of use of a progress exam at the Dalhousie College of Pharmacy
Keywords:
Evaluation, Comprehensive Exam, Pharmacy, Progress ExamAbstract
Introduction: The purpose of this evaluation was to determine if the seven original objectives of the Progress Exam had been achieved, and to provide recommendations for improvements.
Description of Progress Exam: The Progress Exam was comprised of 100 multiple choice questions based on a blueprint. All students wrote the same exam once yearly at the same time.
Methods: A theory-of-change approach to evaluation was used. Data from 1) a document review; 2) an interview; 3) a survey ; and 4) an eleven year evaluation of Progress Exam results, were collated and analysed.
Evaluation: This evaluation indicated that four of the original seven objectives were achieved (knowledge learned was similar in two different curricula; students’ knowledge improved over the duration of the programme; student’s knowledge was retained throughout the programme; and, the Progress Exam provided an opportunity to write a comprehensive exam).
Future Work: Review of the results of this evaluation led to recommendations for improvements that are under consideration by the Curriculum Committee.
References
Albanese, M.A. & Mitchell, S. (1993). Problem-based learning: a review of literature on its outcomes and implementation issues. Academic Medicine, 68, 52-81.
Arnold, L. & Willoughby T.L. (1990). The quarterly profile examination. Academic Medicine, 65, 515-516.
Bird, E., Anderson, H.M., Anaya, G. & Moore, D.L. (2005). Beginning an assessment project: A case study using data audit and content analysis. American Journal of Pharmaceutical Education, 69(3), 356-369.
Blake, J.M., Norman, G.R., Keane, D.R., Mueller, C.B., Cunnington, J. & Didyk, N. (1996). Introducing progress testing in McMaster University's problem-based medical curriculum: psychometric properties and effect on learning. Academic Medicine, 71(9), 1002-1007.
Bloom, B.S. (1956). Taxonomy of Educational Objectives, Handbook I: The Cognitive Domain. New York: MacKay Co Inc.
Canadian Council for Accreditation of Pharmacy Programs [CCAPP]. Accreditation Standards and Guidelines for the First Professional Degree in Pharmacy Programs (online). Available at: http://www.ccapp- accredit.ca/. Accessed 8th February, 2016.
Considine, J., Botti, M. & Thomas, S. (2005). Design, format, validity and reliability of multiple choice questions for use in nursing education and research. Collegian, 12(1), 19-24.
Dolmans, D. & Schmidt, H. (1996). The advantages of problem-based curricula. Postgraduate Medicine, 72, 535-538.
Lunz, M.E. (2014). Examination development guidelines. Measurement Research Associates. Available at: http://www. measurementresearch.com/media/evalguidelines.pdf. Accessed 8th February, 2016.
McHarg, J., Bradley, P., Chamberlain, S., Ricketts, C., Searle, J., & McLachlan, J.C. (2005). Assessment of progress tests. Medical Education, 39(2), 221-227.
Norman, G.R. & Schmidt, H.G. (1992). The psychological basis of problem-based learning: a review of evidence. Academic Medicine, 67, 557-565.
Patton, M.Q. (2008). Utilization-focused evaluation. 4th ed. SAGE Publications, Inc.
Schmidt, H.G. (1983). Problem-based learning: rationale and description. Medical Education, 17, 11-16.
Schuwirth, L.W. & van der Vleuten, C.P. (2012). The use of progress testing. Perspectives on Medical Education, 1(1), 24-30.
Swanson, D.B., Holtzman, K.Z., Butler, A., Langer, M.M., Nelson, M.V., Chow, J.W., Fuller, R., Patterson, J.A. & Boohan, M,. (2010). Collaboration across the pond: the multi-school progress testing project. Medical Teacher, 32(6), 480-485.
Szilagyi, J.E. (2008). Curricular progress assessments: the MileMarker. American Journal of Pharmaceutical Education, 72(5), 101.
Treasury Board of Canada Secretariart. (2012). Theory- Based Approaches to Evaluation: Concepts and Practices (online). Available at: http://www.tbs-sct.gc.ca. Accessed 8th February, 2016.
U.S. Department of Health and Human Services Centers for Disease Control and Prevention. (2011). Officer of the Director, Office of Strategy and Innovation. Introduction to program evaluation for public health programs: A self- study guide. Atlanta GA. Centers for Disease Control and Prevention (online). Available at: www.cdc.gov/eval/ guide/. Accessed 31st October, 2016
Van der Vleuten, C.P.M., Verwijnen, G.M. & Wijnen, W.H.F. (1996). Fifteen years experience with progress testing in a problem-based learning curriculum. Medical Teacher, 18(2), 103-9.
Varma, S. (2014). Preliminary item statistics using point- biserial correlation and p-values. Educational Data Systems Inc. (online). Available at: http://eddata.com/ wpcontent/uploads/2015/11/ EDS-Biserial.pdf. Accessed 8th February, 2016.
Waskiewicz, R.A. (2011). Pharmacy students' test-taking motivation-effort on a low-stakes standardized test. American Journal of Pharmaceutical Education, 75(3), Article 41.
