RESEARCH ARTICLE: Implementation and students’ perception of a criterion-referenced standard setting in a therapeutics course

Authors

  • YeeAnn Chen University of Michigan, USA.
  • Crystal Zhou University of California, San Francisco, USA.
  • Andrew Leeds University of California, San Francisco, USA
  • Jaekyu Shin University of California San Francisco, USA.

DOI:

https://doi.org/10.46542/pe.2020.201.303312

Keywords:

Pharmacy education, assessment, standard setting, criterion-referenced methods, Student stress

Abstract

Objective: To implement a criterion-referenced method to set standards for grading written tests in a didactic course and to assess students’ perceptions of the implementation.

Methods: The Angoff method, a criterion-referenced method, was implemented in a therapeutics course with a letter grading system. Students were surveyed on their perceptions of the method including stress and test anxiety level after the course.

Results: Of 122 students enrolled, 118 responded. More than 60% of respondents felt that the criterion-referenced method was fairer and reflected competency better than a norm-referenced method. The percent of respondents who felt that the new method increased the level of stress and test anxiety was higher than that of those who did not.

Conclusions: A criterion-referenced method was successfully implemented in a pharmacy didactic course with a letter grading system. The implementation was overall favourably received by students although it may have increased the level of stress and test anxiety.

Author Biographies

YeeAnn Chen, University of Michigan, USA.

University of California, San Francisco at time of research

Crystal Zhou, University of California, San Francisco, USA.

Assistant Clinical Professor, Department of Clinical Pharmacy, School of Pharmacy

Andrew Leeds, University of California, San Francisco, USA

Professor Emeritus,  Department of Clinical Pharmacy

Jaekyu Shin, University of California San Francisco, USA.

Professor of Clinical Pharmacy, Department of Clinical Pharmacy

References

Alston, G.L., & Love, B.L. (2010). Development of a reliable, valid annual skills mastery assessment examination. A standardized rubric to evaluate student presentations. American Journal of Pharmaceutical Education, 74(80). https://doi.org/10.5688/aj740580

Beall, J.W., DeHart, R.M., Riggs, R.M., & Hensley, J. (2015). Perceived stress, stressors, and coping mechanisms among Doctor of Pharmacy students. Pharmacy (Basel), 25,344-354. https://doi.org/10.3390/pharmacy3040344

Bendavid, M.R. (2000). AMEE guide No. 18: Standard setting in student assessment. Medical Teacher, 22, 120-130. https://doi.org/10.1080/01421590078526

Fagin, A.P., Howell, T.H., Da Silva, J., & Park, S.E. (2014). The impact on dental students of changes to the National Board Dental Examination grading system. Journal of Dental Education, 78, 813. https://doi.org/10.1002/j.0022-0337.2014.78.6.tb05734.x

Gregory, D.F., Boje, K.M., Carter, R.A., Daugherty, K.K., Hagemeier, N.E., Munger, M.A., Umland, E.M., & Wagner, J.L. (2019). Leading Change in Academic Pharmacy: Report of the 2018-2019 AACP Academic Affairs Committee. American Journal of Pharmaceutical Education, 83, 7661. https://doi.org/10.5688/ajpe7661

Hembree, R. (1988). Correlates, causes, effects, and treatment of test anxiety. Review of Educational Research, 58(1), 47-77. https://doi.org/10.3102/00346543058001047

Hirsch, J.D., Do, A.H., Hollenbach, K.A., Manoguerra, A.S., & Adler, D.S. (2009). Students' health-related quality of life across the preclinical pharmacy curriculum. American Journal of Pharmaceutical Education, 73, 147. https://doi.org/10.5688/aj7308147

Holmboe, E.S., Sherbino, J., Long, D.M., Swing, S.R., & Frank, J.R. (2010). The role of assessment in competency-based medical education. Medical Teacher, 32, 676-682. https://doi.org/10.3109/0142159X.2010.500704

Katajavuori, N., Salminen, O., Vuorensola, K., Huhtala, H., Vuorela, P., & Hirvonen, J. (2017). Competence-Based Pharmacy Education in the University of Helsinki. Pharmacy, 5(29). https://doi.org/10.3390/pharmacy5020029

Katoue, M.G., & Schwinghammer, T.L. (2000). Competency-based education in pharmacy: A review of its development, applications, and challenges. Journal of Evaluation in Clinical Practice, 26, 1114-1123. https://doi.org/10.1111/jep.13362

Koster, A., Schalekamp, T., & Meijerman, I. (2017). Implementation of Competency-Based Pharmacy Education (CBPE). Pharmacy, 5(10). https://doi.org/10.3390/pharmacy5010010

Marshall, L.L., Allison, A., Nykamp, D., & Lanke, S. (2008). Perceived stress and quality of life among doctor of pharmacy students. American Journal of Pharmaceutical Education, 72, 137. https://doi.org/10.5688/aj7206137

Peeters, M.J., Sahloff, E.G., & Stone, G.E. (2010). A standardized rubric to evaluate student presentations. American Journal of Pharmaceutical Education, 74, 171. https://doi.org/10.5688/aj7409171

Pick, A.M., Henriksen, B.S., Hamilton, W.R., & Monaghan, M.S. (2013). Essential information for mentoring students interested in residency training. Current Pharmacy Teaching and Learning, 5, 546-554. https://doi.org/10.1016/j.cptl.2013.07.017

Pincus, K., Hammond, A.D., Reed, B.N., & Feemster, A.A. (2019). Effect of Advanced Pharmacy practice experience grading scheme on residency match rates. American Journal of Pharmaceutical Education, 83, 6735.

https://doi.org/10.5688/ajpe6735

Royal, K.D., & Guskey, T.R. (2015). On the appropriateness of norm- and criterion-referenced assessments in medical education. Ear, Nose & Throat Journal, 94, 252-254. https://doi.org/10.1177/ 014556131509400701

Searle, J. (2000). Defining competency - the role of standard setting. Medical Education, 34, 363-366. https://doi.org/10.1046/j.1365-2923.2000.00690.x

Senthong, V., Chindaprasirt, J., Sawanyawisuth, K., Aekphachaisawat, N., Chaowattanapanit, S., Limpawattana, P., Choonhakarn, C., & Sookprasert, A. (2013). Group versus modified individual standard-setting on multiple-choice questions with the Angoff method for fourth-year medical students in the internal medicine clerkship. Advanced Medical Education Practice, 4, 195-200. https://doi.org/10.2147/AMEP.S46705

Schullo-Feulner, A., Kolar, C., & Janke, K.K. (2015). A Five-Year Evaluation of Examination Structure in a Cardiovascular Pharmacotherapy Course. American Journal of Pharmaceutical Education, 79(98). https://doi.org/10.5688/ajpe79798

Schuwirth, L.W.T., & van der Vleuten, C.P.M. (2014). How to design a useful test: The principles of assessment. In Understanding medical education, (Ed. T. Swanwick) West Sussex, UK: John Wiley & Sons., pp. 243-254. https://doi.org/10.1002/9781118472361.ch18

Shin, J., & Gruenberg, K. (2019). Test-enhanced learning in a pharmacy therapeutics course. Pharmacy Education, 19, 100-107

Shulruf, B., Wilkinson, T., Weller, J., Jones, P., & Poole, P. (2016). Insights into the Angoff method: results from a simulation study. BMC Medical Education, 16, 134. https://doi.org/10.1186/s12909-016-0656-7

Talente, G., Haist, S.A., & Wilson, J.F. (2003). A model for setting performance standards for standardized patient examinations Evaluation & The Health Professions, 26, 427-446. https://doi.org/10.1177/0163278703258105

Wayne, D.B., Cohen, E., Makoul, G., & McGaghie, W.C. (2008). The impact of judge selection on standard setting for a patient survey of physician communication Skills. Academic Medicine, 83(10 Suppl), S17-S20. https://doi.org/10.1097/ACM.0b013e318183e7bd

Yudkowski, R., Downing, S.M., & Tekian, A. (2009). Standard setting. In Assessment in health professions education, (Ed R. Yudkowski & S.M. Downing). New York City, NY: Taylor & Francis, pp. 119-148

Yudkowski, R., Tekian, A., & Downing, S.M. (2006). Procedures for establishing defensible absolute passing scores on performance examinations in health professions education. Teaching and Learning in Medicine, 18, 50-57. https://doi.org/10.1207/s15328015tlm1801_11

Downloads

Published

21-10-2020

How to Cite

Chen, Y., Zhou, C., Leeds, A., & Shin, J. (2020). RESEARCH ARTICLE: Implementation and students’ perception of a criterion-referenced standard setting in a therapeutics course. Pharmacy Education, 20(1), p. 303 – 312. https://doi.org/10.46542/pe.2020.201.303312

Issue

Vol. 20 (2020)

Section

Research Article

License

It is a condition of the publication that authors vest or license copyright in their articles, including abstracts, in FIP. This enables us to ensure full copyright protection and to disseminate the article, and the journal, to the widest possible readership in print and electronic formats as appropriate. Authors may, of course, use the material elsewhere after publication providing that prior permission is obtained from FIP. Authors are themselves responsible for obtaining permission to reproduce copyright material from other sources.