Comparison of pharmacy student, evaluator, and standardised patient assessment of Objective Structured Clinical Examination (OSCE) performance
DOI:
https://doi.org/10.46542/pe.2023.231.314320Keywords:
Assessment, Objective Structured Clinical Examination, OSCEAbstract
Objective: The study aimed to (1) determine if standardised patient (SP) assessment of student communication differs from faculty assessment and (2) identify factors that may affect their assessment scoring since differences between evaluator scoring could impact students’ learning and feedback.
Methods: Students completed four OSCEs in spring 2022. Communication skills were scored using a global assessment rubric completed by a faculty evaluator, SP, and student. SPs and students rated their confidence in student recommendations on a scale of 1-5 (1=lowest; 5=highest). Spearman correlation and Wilcoxon matched-pairs signed rank tests were used for analysis.
Results: SP global assessment scores were statistically significantly higher than faculty scores. SP global assessment and confidence scores were strongly correlated.
Conclusion: Differences in OSCE scoring among evaluator types and factors that may impact scoring are important to explore. SP scores were consistently higher than faculty scores, which should be considered when using OSCEs as assessments.
References
Barrickman, A. L., & Maynor, L. (2021). PROGRAMME DESCRIPTION: Implementation of a capstone course with Formative OSCEs to evaluate preparedness for Advanced Pharmacy Practice Experiences (APPE). Pharmacy Education, 21, 145–158. https://doi.org/10.46542/pe.2021.211.145158
Chisnall, B., Vince, T., Hall, S., & Tribe, R. (2015). Evaluation of outcomes of a formative objective structured clinical examination for second-year UK medical students. International journal of medical education, 6, 76–83. https://doi.org/10.5116/ijme.5572.a534
Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Routledge. https://doi.org/10.4324/9780203771587
Deng, B., Fenn, N. E., 3rd, & Plake, K. S. (2019). Impact of a teaching objective structured clinical examination (TOSCE) on student confidence in a pharmacy skills laboratory. Currents in pharmacy teaching & learning, 11(2), 145–154. https://doi.org/10.1016/j.cptl.2018.11.009
Ferreira, É. M. R., Pinto, R. Z., Arantes, P. M. M., Vieira, É. L. M., Teixeira, A. L., Ferreira, F. R., & Vaz, D. V. (2020). Stress, anxiety, self-efficacy, and the meanings that physical therapy students attribute to their experience with an objective structured clinical examination. BMC medical education, 20(1), 296. https://doi.org/10.1186/s12909-020-02202-5
Gillette, C., Stanton, R. B., Rockich-Winston, N., Rudolph, M., & Anderson, H. G., Jr (2017). Cost-Effectiveness of Using Standardized Patients to Assess Student-Pharmacist Communication Skills. American journal of pharmaceutical education, 81(10), 6120. https://doi.org/10.5688/ajpe6120
Korayem, G. B., Alshaya, O. A., Kurdi, S. M., Alnajjar, L. I., Badr, A. F., Alfahed, A., & Cluntun, A. (2022). Simulation-Based Education Implementation in Pharmacy Curriculum: A Review of the Current Status. Advances in medical education and practice, 13, 649–660. https://doi.org/10.2147/AMEP.S366724
Kristina, S. A., & Wijoyo, Y. (2019). Assessment of Pharmacy Students? Clinical Skills using Objective Structured Clinical Examination (OSCE): A Literature Review. Systematic Reviews in Pharmacy, 10(1), 55–60. https://doi.org/10.5530/srp.2019.1.9
McLaughlin, K., Gregor, L., Jones, A., & Coderre, S. (2006). Can standardized patients replace physicians as OSCE examiners?. BMC medical education, 6, 12. https://doi.org/10.1186/1472-6920-6-12
Ragan, R. E., Virtue, D. W., & Chi, S. J. (2013). An assessment program using standardized clients to determine student readiness for clinical practice. American journal of pharmaceutical education, 77(1), 14. https://doi.org/10.5688/ajpe77114
Salinitri, F. D., O'Connell, M. B., Garwood, C. L., Lehr, V. T., & Abdallah, K. (2012). An objective structured clinical examination to assess problem-based learning. American journal of pharmaceutical education, 76(3), 44. https://doi.org/10.5688/ajpe76344
Schwartzman, E., Hsu, D. I., Law, A. V., & Chung, E. P. (2011). Assessment of patient communication skills during OSCE: examining effectiveness of a training program in minimizing inter-grader variability. Patient education and counseling, 83(3), 472–477. https://doi.org/10.1016/j.pec.2011.04.001
Shirwaikar A. (2015). Objective structured clinical examination (OSCE) in pharmacy education - a trend. Pharmacy practice, 13(4), 627. https://doi.org/10.18549/PharmPract.2015.04.627
Smithson, J., Bellingan, M., Glass, B., & Mills, J. (2015). Standardized patients in pharmacy education: An integrative literature review. Currents in Pharmacy Teaching and Learning, 7(6), 851–863. https://doi.org/10.1016/j.cptl.2015.08.002
Sturpe D. A. (2010). Objective structured clinical examinations in doctor of pharmacy programs in the United States. American journal of pharmaceutical education, 74(8), 148. https://doi.org/10.5688/aj7408148
Urteaga, E. M., Attridge, R. L., Tovar, J. M., & Witte, A. P. (2015). Evaluation of Clinical and Communication Skills of Pharmacy Students and Pharmacists with an Objective Structured Clinical Examination. American journal of pharmaceutical education, 79(8), 122. https://doi.org/10.5688/ajpe798122
Weaver, S. B., Daftary, M., Wingate, L., & Turner, M. (2022). Evaluation of faculty inter-variability OSCE grade scoring on overall student performance in a laboratory course. Pharmacy Education, 22(1), 48–53. https://doi.org/10.46542/pe.2022.221.4853
