RESEARCH ARTICLE: Comprehensive assessment of reliability and validity for the clinical cases in simulated community pharmacy

Authors

DOI:

https://doi.org/10.46542/pe.2022.221.183190

Keywords:

Community pharmacy, Objective structured clinical examinations, Reliability, Simulation, Validity

Abstract

Background: The objective structured clinical examination (OSCE) is used to measure the clinical competence of pharmacy students in a community pharmacy setup. However, the OSCE needs to be standardised to assess the clinical competence of the student accurately.   

Objectives: The present study was aimed to assess the reliability and validity of two clinical cases used in the simulated community pharmacy.  

Methods: OSCE simulation was performed by the students with two clinical cases in a simulated community pharmacy. The reliability was measured using Cronbach’s α and Mc Donald’s ω. Exploratory factor analysis (EFA) and Confirmatory factor analysis (CFA) were used to measure the validity of the cases.   

Results: Among the two cases, the first case scenario was found to have a good model fit. However, the second case scenario has a poor model fit which was determined by the CFA. The inadequate sample size and factor loading in EFA were the main reasons for poor model fit in the second case scenario.   

Conclusion: The internal consistency, sample adequacy, factor loading, test for an exact fit, and fit measurements should be ensured for the clinical cases included in OSCE. This will help the academician to ensure the accurate assessment of the clinical competence of the student in a simulated community pharmacy. 

Author Biography

Palanisamy Amirthalingam, University of Tabuk, Tabuk, Saudi Arabia

Department of Pharmacy Practice & Pharmacy Simulation Unit

References

Abdi, A.M., Meštrović, A., Demirdamar, R., & Basgut, B. (2019). Preparing competent graduates for delivering pharmaceutical care: an experience from Northern Cyprus. BMC medical education, 19(1), 442. https://doi.org/10.1186/s12909-019-1875-5

Achary, A., & Khanal D.P. (2020). Compliance on Good Pharmacy Practice (GPP) Guideline in Community Pharmacy of Western Nepal, Nepalgunj. Journal of Pharmacy Practice,3 No.2,25

Alavi, M., Visentin, D.C., Thapa, D.K., Hunt, G.E., Watson, R., & Cleary, M. (2020). Chi-square for model fit in confirmatory factor analysis. Journal of advanced nursing, 76(9), 2209-2211. https://doi.org/10.1111/jan.14399

Al-Haqan, A., Al-Taweel, D., Koshy, S., & Alghanem, S. (2021). Evolving to Objective Structured Clinical Exams (OSCE): Transitional experience in an undergraduate pharmacy program in Kuwait. Saudi pharmaceutical journal, 29(1), 104–113. https://doi.org/10.1016/j.jsps.2020.12.013

Al-Jedai, A., Qaisi, S., & Al-Meman, A. (2016). Pharmacy Practice and the Health Care System in Saudi Arabia. The Canadian journal of hospital pharmacy, 69(3), 231–237. https://doi.org/10.4212/cjhp.v69i3.1561

Almaghaslah, D., Alsayari, A., Almanasef, M., & Asiri, A. (2021). A Cross-Sectional Study on Pharmacy Students' Career Choices in the Light of Saudi Vision 2030: Will Community Pharmacy Continue to Be the Most Promising, but Least Preferred, Sector? International journal of environmental research and public health, 18(9), 4589. https://doi.org/10.3390/ijerph18094589

Andrew, F.H., Jacob, J.C. (2020). Use Omega Rather than Cronbach’s Alpha for Estimating Reliability. But…, Communication Methods and Measures, 14(1), 1-24. https://doi.org/10.1080/19312458.2020.1718629

Badro, D.A., Sacre, H., Hallit, S., Amhaz, A., & Salameh, P. (2020). Good pharmacy practice assessment among community pharmacies in Lebanon. Pharmacy practice, 18(1), 1745. https://doi.org/10.18549/PharmPract.2020.1.1745

Bartlett, M.S. (1954) A Note on the Multiplying Factors for Various χ2 Approximations. Journal of the Royal Statistical Society: Series B, 16, 296–8. https://doi.org/10.1111/j.25176161.1954.tb00174.x

Brennan, P.A., Konieczny, K., Groves, J., Parker, M., Sherman, K. P., Foulkes, J., Hills, S., & Featherstone, C. (2015). Development, validation and initial outcomes of a questionnaire to examine human factors in postgraduate surgical objective structured clinical examinations. The British journal of surgery, 102(4), 423–430. https://doi.org/10.1002/bjs.9721

Chesser, A.M., Laing, M.R., Miedzybrodzka, Z.H., Brittenden, J., & Heys, S.D. (2004). Factor analysis can be a useful standard setting tool in a high stakes OSCE assessment. Medical education, 38(8), 825–831. https://doi.org/10.1111/j.1365-2929.2004.01821.x

Costello, A.B., Osborne, J. (2005). Best Practices in Exploratory Factor Analysis: Four Recommendations for Getting the Most from Your Analysis. Practical Assessment Research & Evaluation, 10,1-9

Dagnall, N., Denovan, A., Parker, A., Drinkwater, K., & Walsh, R. S. (2018). Confirmatory Factor Analysis of the Inventory of Personality Organization-Reality Testing Subscale. Frontiers in psychology, 9, 1116. https://doi.org/10.3389/fpsyg.2018.01116

Deng, L., & Chan, W. (2017). Testing the Difference Between Reliability Coefficients Alpha and Omega. Educational and psychological measurement, 77(2), 185–203. https://doi.org/10.1177/0013164416658325

Dizon, S., Malcolm, J.C., Rethans, J.J., & Pugh, D. (2021). Assessing the validity of an OSCE developed to assess rare, emergent or complex clinical conditions in endocrinology & metabolism. BMC medical education, 21(1), 288. https://doi.org/10.1186/s12909-021-02653-4

Flora, D.B. (2020) Your Coefficient Alpha Is Probably Wrong, but Which Coefficient Omega Is Right? A Tutorial on Using R to Obtain Better Reliability Estimates. Advances in Methods and Practices in Psychological Science, 484-501. https://doi.org/10.1177/2515245920951747

Furmedge, D.S., Smith, L.J., & Sturrock, A. (2016). Developing doctors: what are the attitudes and perceptions of year 1 and 2 medical students towards a new integrated formative objective structured clinical examination?. BMC medical education, 16, 32. https://doi.org/10.1186/s12909-

Guadagnoli, E., & Velicer, W.F. (1988). Relation of sample size to the stability of component patterns. Psychological bulletin, 103(2), 265–275. https://doi.org/10.1037/0033-

Hancock, G.R., & French, B.F. (2013). Power Analysis in Structural Equation Modeling. In G.R. Hancock, & R.O. Mueller (Eds.), Structural Equation Modeling: A Second Course (2nd ed., pp. 117-159)

Htay, M., Shrivastava, S., Sahoo, S. (2019) 'Development and Validation of Medical Students’ Professionalism Assessment Scale (MSPAS) In OSCE: Simulated Patients’ rating scale', MedEdPublish, 8(2), 24. https://doi.org/10.15694/mep.2019.000094.1

Jackson, D.L. (2003). Revisiting Sample Size and Number of Parameter Estimates: Some Support for the N:q Hypothesis. Structural Equation Modeling, 10, 128-141. https://doi.org/10.1207/S15328007SEM1001_6

Kaiser, H.F. (1974). An index of factorial simplicity. Psychometrika, 39, 31–6. https://doi.org/10.1007/BF02291575

Knekta, E., Runyon, C., & Eddy, S. (2019). One Size Doesn't Fit All: Using Factor Analysis to Gather Validity Evidence When Using Surveys in Your Research. CBE life sciences education, 18(1). https://doi.org/10.1187/cbe.18-04-0064

Kline, R.B. (2005). Principles and practice of structural equation modeling (2nd ed.). New York, NY: Guilford Press

Kyriazos, T. (2018). Applied Psychometrics: Sample Size and Sample Power Considerations in Factor Analysis (EFA, CFA) and SEM in General. Psychology, 9, :2207-2230. https://doi.org/10.4236/psych.2018.98126

Lee, T., Cai, L., & MacCallum, R. (2012). Power Analysis for Tests of Structural Equation Models. In R. H. Hoyle (Ed.), Handbook of Structural Equation Modeling (pp. 181-194). New York: Guilford Press

Lim, A.S., Lee, S., Karunaratne, N., & Caliph, S. (2020). Pharmacy Students' Perceptions and Performance on the Use of an Online Virtual Experience Tool for Practicing Objective Structured Clinical Examinations. American journal of pharmaceutical education, 84(11), 7920. https://doi.org/10.5688/ajpe7920

Liou, C.Y., Musicus, B.R. (2008). "Cross Entropy Approximation of Structured Gaussian Covariance Matrices". IEEE Transactions on Signal Processing, 56 (7), 3362–3367. https://doi.org/10.1109/TSP.2008.917878

Majumder, M., Kumar, A., Krishnamurthy, K., Ojeh, N., Adams, O.P., & Sa, B. (2019). An evaluative study of objective structured clinical examination (OSCE): students and examiners perspectives. Advances in medical education and practice, 10, 387–397. https://doi.org/10.2147/AMEP.S197275

Paul F.Wimmers., & Guido F.Schauer. (2017). Validating OSCE Performance: The Impact of General Intelligence. Health Professions Education, 3(2), 79-84. https://doi.org/10.1016/j.hpe.2016.12.002

Peeters, M.J., Cor, M.K., Petite, S.E., & Schroeder, M.N. (2021). Validation evidence using generalizability theory for an objective structured clinical examination. Innovations in pharmacy, 12(1). https://doi.org/10.24926/iip.v12i1.2110

Peterson, R.A. (2000). A Meta-Analysis of Variance Accounted for and Factor Loadings in Exploratory Factor Analysis. Marketing Letters, 11, 261–275. https://doi.org/10.1023/A:1008191211004

Piumatti, G., Cerutti, B., & Perron, N.J. (2021). Assessing communication skills during OSCE: need for integrated psychometric approaches. BMC medical education, 21(1), 106. https://doi.org/10.1186/s12909-021-02552-8

Rasheed, M.K., Alqasoumi, A., Hasan, S.S., & Babar, Z.U. (2020). The community pharmacy practice change towards patient-centered care in Saudi Arabia: a qualitative perspective. Journal of pharmaceutical policy and practice, 13, 59. https://doi.org/10.1186/s40545-020-00267-7

Ravinder, B., Saraswathi, A.B. (2020). Literature Review Of Cronbach alpha coefficient (Α) And Mcdonald's Omega Coefficient (Ω). European Journal of Molecular & Clinical Medicine, 7(6), 2943-2949

Said Said Elshama. (2021). How to Design and Apply an Objective Structured Clinical Examination (OSCE) in Medical Education? Iberoamerican Journal of Medicine, 1, 51-55. https://doi.org/10.53986/ibjm.2021.0009

Savage, A., Minshew, L.M., Anksorus, H.N., & McLaughlin, J. E. (2021). Remote OSCE Experience: What First Year Pharmacy Students Liked, Learned, and Suggested for Future Implementations. Pharmacy, 9(1), 62. https://doi.org/10.3390/pharmacy9010062

Schumacker, R.E., & Lomax, R.G. (2015). A Beginner’s Guide to Structural Equation Modeling (4th ed.). New York, Routledge. https://doi.org/10.4324/9781315749105

Shirwaikar, A. (2015). Objective structured clinical examination (OSCE) in pharmacy education - a trend. Pharmacy practice, 13(4), 627. https://doi.org/10.18549/PharmPract.2015.04.627

Susi, A.K., & Yosef, W. (2019) Assessment of Pharmacy Students’ Clinical Skills using Objective Structured Clinical Examination (OSCE): A Literature Review. Systematic Reviews in Pharmacy, 10(1), 55-60. https://doi.org/10.5530/srp.2019.1.9

Viladrich, C., Angulo-Brunet, A., & Doval, E. (2017). A Journey around alpha and omega to estimate internal consistency reliability. Anales de Psicología, 33(3), :755–782. https://doi.org/10.6018/analesps.33.3.268401

Wang, L.L., Watts, A.S., Anderson, R.A., & Little, T.D. (2013). Common Fallacies in Quantitative Research Methodology. In T.D. Little (Ed.), The Oxford Handbook of Quantitative Methods (pp. 718-758). https://doi.org/10.1093/oxfordhb/9780199934898.013.0031

Watkins, M.W. (2018). Exploratory Factor Analysis: A Guide to Best Practice. Journal of Black Psychology, 44(3), 219-246. https://doi.org/10.1177/0095798418771807

World Health Organization. (2011). Joint FIP/WHO Guidelines on Good Pharmacy Practice:Standards for quality of pharmacy services. Geneva

Yazbeck Karam, V., Park, Y.S., Tekian, A., & Youssef, N. (2018). Evaluating the validity evidence of an OSCE: results from a new medical school. BMC medical education, 18(1), 313. https://doi.org/10.1186/s12909-018-1421-x

Downloads

Published

2022-03-11

How to Cite

Amirthalingam, P. (2022). RESEARCH ARTICLE: Comprehensive assessment of reliability and validity for the clinical cases in simulated community pharmacy . Pharmacy Education, 22(1), p. 183–190. https://doi.org/10.46542/pe.2022.221.183190

Issue

Section

Research Article