Evaluation of faculty inter-variability OSCE grade scoring on overall student performance in a laboratory course

Authors

DOI:

https://doi.org/10.46542/pe.2022.221.4853

Keywords:

Faculty, Inter-variability, Objective Structured Clinical Examination (OSCE), Performance

Abstract

Introduction: Objective structured clinical examinations (OSCEs) are considered the gold standard for evaluating pharmacy students’ clinical skills due to their reliability and validity. 

Aim: The purpose of this study was to determine whether faculty inter-variability in OSCE grading had a significant impact on a student’s overall performance. 

Methods: A retrospective analysis was conducted using data from two cohorts of third-year pharmacy students. Descriptive statistics, simple linear regression, and multivariate linear regression analyses were conducted. 

Results: There were 120 students that participated in the OSCE with a mean score of 66.7%.    Higher scores in the Integrated Therapeutics (IT) 2 lecture series and the IT 2 lab course corresponded to better OSCE scores.  Out of 17 evaluators, six were found to rate students significantly lower and one was found to rate students significantly higher in comparison to a reference evaluator who evaluated students closest to the overall mean. 

Conclusion: It is likely that standardised grading, and possibly additional training, may be needed to ensure a fair and appropriate evaluation of OSCE performance.

Author Biographies

Salome Bwayo Weaver , Howard University College of Pharmacy, Washington DC, United States

  

Monika Daftary, Howard University College of Pharmacy, Washington DC, United States

  

La'Marcus Wingate, Howard University College of Pharmacy, Washington DC, United States

  

Malaika Turner, Howard University College of Pharmacy, Washington DC, United States

  

References

Ali, M, Pawluk, S.A., Rainkie, D.C., & Wilby, K,J. (2019) Pass-Fail decisions for borderline performers after a summative objective structured clinical examination. American Journal of Pharmacy Education, 83(2), Article 6849. https://doi.org/10.5688/ajpe6849

Barman, A. (2005) Critiques on the objective structured clinical examination. Annals – Academic Medicine of Singapore, 34(8), 478–482

Bartman, I., Smee, S., & Roy, M. (2013). A method for identifying extreme OSCE examiners. The clinical teacher, 10 (1), 27-31. https://doi.org/10.1111/j.1743-498X.2012.00607.x

Binh, P.D., An, P.L., Tuan, T.D., & Leppink, J.(2018). Inter-Rater reliability of a professionalism OSCE developed in family medicine training University of Medicine and Pharmacy. MedPharmRes, 2(1), 20-24. https://doi.org/10.32895/UMP.MPR.2.1.20

Brannick, M.T., Erol-Korkmaz, H.T., & Prewett, M. (2011). A systematic review of the reliability of objective structured clinical examination scores. Medical Education, 45(12), 1181-1189. https://doi.org/10.1111/j.1365-2923.2011.04075.x

Cauthon, K.A., Attridge, R.L., Urteaga, E.M., Witte, A.P. (2017) An Objective Structured Clinical Examination to Assess Pharmacy Resident Performance. Innovations in Pharmacy, 8(2), Article 5. https://doi.org/10.24926/IIP.V812.510

Corbo, M., Patel, J.P., Abdel Tawab R, Davies JG. (2006). Evaluating clinical skills of undergraduate pharmacy students using objective structured clinical examinations (OSCEs). Pharmacy Education. 6, 53–58. https://doi.org/10.1080/15602210500528372

Curtis, S., Smith, K.J., & Taylor, J. (2019) Implementation of objective structured clinical examinations (OSCE) in a doctor of pharmacy program. Currents in Pharmacy Teaching and Learning, 11, 832-837. https://doi.org/10.1016/j.cptl.2019.04.011

Duanmu, Y, Henwood, P.C., Takhar, S.S., Chan, W., Rempell, J.S., Liteplo, A.S., Koskenoja, V., Noble, V.E., & Kimberly, H.H. (2019). Correlation of OSCE performance and point-of-care ultrasound scans numbers among a cohort of emergency medicine residents. The ultrasound Journal, 11(1):3. https://doi.org/10.1186/s13089-019-0118-7

Kilminster, S., Roberts, T., Morris, P. (2007). Incorporating patients’ assessment into objective structured clinical examinations. Education for Health, 20(6)

Kirton, S.B., & Kravitz, L. (2011). Objective Structured Clinical Examinations (OSCEs) Compared With Traditional Assessment Methods. American Journal of Pharmacy Education, 75(6), Article 111. https://doi.org/10.5688/ajpe756111

Mafinejad, M.K., Rastegarpanah, M, Moosavi, F, Shirazi, M. (2017). Training and Validation of Standardized Patients for Assessing Communication and Counseling Skills of Pharmacy Students: A Pilot Study. Journal of Research in Pharmacy Practice, 6(2), 83-88. https://doi.org/10.4103/jrpp.JRPP_17_20

Reid, K, Smallwood, D, Collins, M, Sutherland, R, Dodds, A. (2016). Taking OSCE examiner training on the road: reaching the masses. Medical Education Online, 21. https://doi.org/10.3402/meo.v21.32389

Salinitri, F.D., O'Connell, M.B., Garwood, C.L., Lehr, V.T., & Abdallah, K. (2012). An objective structured clinical examination to assess problem-based learning. American Journal of Pharmacy Education, 76(3), 44. https://doi.org/10.5688/ajpe76344

Shahzad, A., Saeed, M.H.B., Paiker, S. (2017). Dental students' concerns regarding OSPE and OSCE: a qualitative feedback for process improvement. BDJ Open. 9, 3, 17009. https://doi.org/10.1038/bdjopen.2017.9

Shirwaikar, A. Objective structured clinical examination (OSCE) in pharmacy education - a trend. (2015). Pharmacy Practice, 13(4), 627. https://doi.org/10.18549/PharmPract.2015.04.627

Sobh, A.H., Austin, Z, Izham, M.I.M., Diab, M.I., Wilby, K.J. (2017). Application of a Systematic Approach to Evaluating Psychometric Properties of a Cumulative Exit-From-Degree Objective Structured Clinical Examination (OSCE). Currents in Pharmacy Teaching and Learning, 9(6), 1091–1098. https://doi.org/10.1016/j.cptl.2017.07.011

Solà, M., Pulpón, A.M., Morin, V., Sancho, R., Clèries, X., Fabrellas, N. (2017). Towards the implementation of OSCE in undergraduate nursing curriculum: A qualitative study. Nurse Education Today. 49, 163-167. https://doi.org/10.1016/j.nedt.2016.11.028. 7

Stowe, C.D., & Gardner, S.F. (2005). Real-Time Standardized Participant Grading of an Objective Structured Clinical Examination. American Journal of Pharmacy Education, 69(3) Article 41. https://doi.org/10.5688/aj690341

Sturpe, D.A. (2010). Objective structured clinical examinations in the doctor of pharmacy programs in the United states. American Journal of Pharmacy Education, 74(8) Article 148. https://doi.org/10.5688/aj7408148

Tamblyn, R.M., Klass, D.J., Schnabl, G.K., & Kopelow, M.L. (1991). Sources of unreliability and bias in standardized‐patient rating, Teaching and Learning in Medicine, 3(2), 74-85. https://doi.org/10.1080/10401339109539486

Vanka, A., Wali, O., Akondi, B.R., Vanka, S., Ravindran, S. (2018). OSCE: A New Assessment Method for Pharmaceutical Education. Indian Journal of Pharmaceutical Education and Research, 52(4s), S1-S6. https://doi.org/10.5530/ijper.52.4s.68

Downloads

Published

08-01-2022

How to Cite

Weaver , S. B. ., Daftary, M. ., Wingate, L., & Turner, M. (2022). Evaluation of faculty inter-variability OSCE grade scoring on overall student performance in a laboratory course. Pharmacy Education, 22(1), p. 48–53. https://doi.org/10.46542/pe.2022.221.4853

Issue

Vol. 22 No. 1 (2022)

Section

Research Article

License

It is a condition of the publication that authors vest or license copyright in their articles, including abstracts, in FIP. This enables us to ensure full copyright protection and to disseminate the article, and the journal, to the widest possible readership in print and electronic formats as appropriate. Authors may, of course, use the material elsewhere after publication providing that prior permission is obtained from FIP. Authors are themselves responsible for obtaining permission to reproduce copyright material from other sources.