REVIEW: Integrated simulation-based skills assessment for evaluating pharmacist competence: A scoping review
Keywords:
Integrated Skills Assessment, Simulation, Health Professionals, PharmacyAbstract
Background: Integrated assessment of health professional competence more closely represents performance of tasks in clinical practice, than assessment models that evaluate fragmented competencies. Simulation continues to expand the range of assessment methods available for training healthcare professionals, however, there is limited understanding of the potential use of integrated simulation-based assessments in pharmacy education.
Objective: To review the range of integrated simulation-based assessments used across the healthcare professions, and explore their applicability to pharmacy education.
Methods: Medline, International Pharmaceutical Abstracts (IPA), EMBASE, CINAHL, ERIC and Scopus were searched for articles in English published between January 1975 and May 2019, to identify studies that reported on the use of integrated simulation-based assessments in medicine, nursing, allied health and pharmacy.
Results: Searches identified 2, 440 records for screening; 59 full-text articles were screened for eligibility, with a total of 20 studies meeting the inclusion criteria for the review. The findings show evidence for a variety of simulation approaches to enable the evaluation of integrated competencies, however the validity and reliability of such approaches is less well documented. Most studies have been conducted in medicine, nursing and allied health with only two studies conducted in pharmacy. All studies used multidimensional assessment instruments but only fourteen were tested for reliability and validity.
Conclusion: Despite the limited number of published studies specifically pertaining to integrated simulation-based assessment in pharmacy, the current approach in other healthcare disciplines provides support for future application in pharmacy.
References
Aggarwal, R., Mytton, O., Derbrew, M., Hananel, D., Heydenberg, M., Barry Issenberg, S., MacAulay, C., Mancini, M.E., Morimoto, T., Soper, N., Ziv, A. & Reznick, R. (2010). Training and simulation for patient safety. Journal of Quality and Safety in Health Care, 19(Suppl. 2), i34-i43
Auffermann, W., Henry, T., Little, B., Tigges, S. & Tridandapani, S. (2015). Simulation for Teaching and Assessment of Nodule Perception on Chest Radiography in Nonradiology Health Care Trainees. Journal of the American College of Radiology, 12(11)
Australian Pharmacy Council. (2017). Intern Year Blueprint Literature Review Canberra, ACT: Australian Pharmacy Council Limited
Barman, A. (2005). Critiques on the objective structured clinical examination. Annals of the Academy of Medicine, 34, 478-482
Benedict, N., Smithburger, P., Calabrese, A., Empey, P., Kobulinsky, L., Seybert, A., Waters, T., Drab, S., Lutz, J., Farkas, D. & Meyer, S. (2017). Blended Simulation Progress Testing for Assessment of Practice Readiness. American Journal of Pharmaceutical Education, 81(1), 1-13 Article 14
Boulet, J. (2008). Summative Assessment in Medicine: The Promise of Simulation for High-stakes Evaluation. Academic Emergency Medicine, 15, 1017-1024. doi:10.1111/j.1553- 2712.2008.00228.x
Boulet, J., Murray, D., Kras, J., Woodhouse, J., McAllister, J. & Ziv, A. (2003). Relaibility and validity of a simulation-based acute care skills assessment for medical students and residents. Anesthesiology, 99(6), 1270-1280
Boulet, J.R. (2008). Summative assessment in medicine: the promise of simulation for high-stakes evaluation. Academic Emergency Medicine, 15(11), 1017-1024. doi:http://dx.doi.org/10.1111/j.1553-2712.2008.00228.x
Boulet, J.R., Murray, D., Kras, J., Woodhouse, J., McAllister, J. & Ziv, A. (2003). Reliability and validity of a simulation-based acute care skills assessment for medical students and residents. Anesthesiology, 99(6), 1270-1280
Bray, B.S., Schwartz, C.R., Odegard, P.S., Hammer, D.P. & Seybert, A.L. (2011). Assessment of human patient simulation-based learning. American Journal of Pharmaceutical Education, 75(10), 208. doi:http://dx.doi.org/10.5688/ajpe7510208
Bujack, L., McMillan, M., Dwyer, J. & Hazelton, M. (1991). Assessing comprehensive nursing performance: the Objective Structured Clinical Assessment (OSCA). Part 2 - Report of the evaluation project. Nurse Education Today, 11, 248-255
DeMaria, S.Jr., Samuelson, S.T., Schwartz, A.D., Sim, A.J. & Levine, A.I. (2013). Simulation-based assessment and retraining for the anesthesiologist seeking reentry to clinical practice: a case series. Anesthesiology, 119(1), 206-217. doi:http://dx.doi.org/10.1097/ALN.0b013e31829761c8
Douglas, H., Raban, M., Walter, S. & Westbrook, J. (2017). Improving our understanding of multi-tasking in healthcare: Drawing together the cognitive psychology and healthcare literature. Applied Ergonomics, 59, 45-55
East, L., Peters, K., Halcomb, E., Raymond, D. & Salamonson, Y. (2014). Evaluating Objective Structured Clinical Assessment (OSCA) in undergraduate nursing. Nurse Education in Practice, 14, 461-467
Epstein, R.M. (2007). Assessment in medical education. New England Journal of Medicine, 356(4), 387-396
Epstein, R.M. & Hundert, E.M. (2002). Defining and assessing professional competence. Journal of the American Medical Association, 287, 226-235
Gimpel, J., Boulet, D. & Errichetti, A. (2003). Evaluating the clinical skills of osteopathic medical students. Journal of Americal Osteopathy Association, 103(6), 267-279
Gupta, P., Dewan, P. & Singh, T. (2010). Objective Structured Clinical Examination (OSCE) Revisited. Medical Education, 47, 911-919
Hirsch, A.C. & Parihar, H.S. (2014). A capstone course with a comprehensive and integrated review of the pharmacy curriculum and student assessment as a preparation for advanced pharmacy practice experiences. American Journal of
Pharmaceutical Education, 78(10), 192. doi:http://dx.doi.
org/10.5688/ajpe7810192
Ilgen, J., Ma, I.W.Y., Hatala, R. & Cook, D.A. (2015). A systematic review of validity evidence for checklists versus global rating scales in simulation-based assessment. Medical Education, 49(2), 161-173
Isenberg, G.A., Berg, K.W., Veloski, J.A., Berg, D.D., Veloski, J.J. & Yeo, C.J. (2011). Evaluation of the use of patient-focused simulation for student assessment in a surgery clerkship. American Journal of Surgery, 201(6), 835-840. doi:http://dx.doi.org/10.1016/j.amjsurg.2010.01.034
Judd, B., Scanlan, J., Alison, J., Waters, D. & Gordon, C. (2016). The validity of a professional competence tool for physiotherapy students in simulation-based clinical education: a Rasch analysis. BMC Medical Education, 16(196), 1-10
Kesten, K.S., Brown, H.F. & Meeker, M.C. (2015). Assessment of APRN Student Competency Using Simulation: A Pilot Study. Nursing Education Perspectives, 36(5), 332-334
Khan, K., Ramachandran, S., Gaunt, K. & Pushkar, P. (2013). The Objective Structured Clinical Examination (OSCE): AMEE Guide No. 81. Part I: An historical and theoretical perspective. Medical Teacher, 35, e1437-1446
Kneebone, R., Nestel, D., Yadollahi, F., Brown, R., Nolan, C., Durack, J., Brenton, H., Moulton, C., Archer, J. & Darzi, A. (2006). Assessing procedural skills in context: Exploring the feasibility of an Integrated Procedural Performance Instrument (IPPI). Medical Education, 40(11), 1105-1114
Kreiter, C.D., Haugen, T., Leaven, T., Goerdt, C., Rosenthal, N., McGaghie, W.C. & Dee, F. (2011). A report on the piloting of a novel computer-based medical case simulation for teaching and formative assessment of diagnostic laboratory testing. Medical Education Online, 16.
doi:http://dx.doi.org/10.3402/meo.v16i0.5646
Lammers, R., Byrwa, M., Fales, W. & Hale, R. (2009). Simulation-based assessment of paramedic pediatric resuscitation skills. Journal of Prehospital Emergency Care, 13(3), 345-356
LeBlanc, V.R., Tabak, D., Kneebone, R., Nestel, D., MacRae, H. & Moulton, C.-A. (2009). Psychometric properties of an integrated assessment of technical and communication skills. The American Journal of Surgery, 197, 96-101
Lipner, R., Messenger, J., Kangilaski, R., Baim, D., Holmes, D., Williams, D. & King, S. (2010). A technical and cognitive skills eveluation of performance in interventional cardioogy procedures using medical simulation. Journal for Simulation in Healthcare, 5(2), 65-74
McBride, M., Waldrop, W., Fehr, J., Boulet, J. & Murray, D. (2011). Simulation in pediatrics: the reliability and validity of a multiscenario assessment. Pediatrics, 128(2), 335-343
McGaghie, W.C., Issenberg, S.B., Petrusa, E.R. & Scalese, R.J. (2010). A critical review of simulation-based medical education research: 2003-2009. Medical Education, 44(1), 50-63. doi:http://dx.doi.org/10.1111/j.1365-2923.2009. 03547.x
Medina, M., Plaza, C., Stowe, C., Robinson, E., DeLander, G., Beck, D., Melchert, R.B., Supernaw, R.B., Roche, V.F., Gleason, B.L., Strong, M.N., Bain, A., Meyer, G.E., Dong, B.J., Rochon, J. & Johnston, P. (2013). Center for the Advancement of Pharmacy Education 2013 Educational Outcomes. American Journal of Pharmaceutical Education, 77(8 ), Article 162
Mikasa, A., Cicero, T. & Adamson, K. (2013). Outcome-Based Evaluation Tool to Evaluate Student Performance in High-Fidelity Simulation. Clinical Simulation in Nursing, 9(9), e361-e367
Mitchell, M., Henderson, A., Groves, M., Dalton, M. & Nulty, D. (2009). The objective structured clinical examination (OSCE): Optimising its value in the undergraduate nursing curriculum. Nurse Education Today, 29, 398-404
Moher, D., Liberati, A., Tetzlaff, J. & Altman, D. (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Annals of Internal Medicine, 151(4), 264-269
Mudumbai, S.C., Gaba, D.M., Boulet, J.R., Howard, S.K. & Davies, M.F. (2012). External validation of simulation-based assessments with other performance measures of third-year anesthesiology residents. Simulation in Healthcare: The Journal of The Society for Medical Simulation, 7(2), 73-80. doi:http://dx.doi.org/10.1097/SIH.0b013e31823d018a
Panzarella, K. & Manyon, A. (2007). A Model for Integrated Assessment of Clinical Competence. Journal of Allied Health, 36(3), 157-164
Pharmaceutical Society of Australia. (2017). Dispensing Practice Guidelines Canberra, ACT: Pharmaceutical Society of Australia Ltd.
Pittenger, A., Chapman, S., Frail, C., Moon, J., Undeberg, M. & Orzoff, J. (2016). Entrustable Professional Activities for Pharmacy Practice. American Journal of Pharmaceutical Education, 80(4), Article 57
Purkerson, D.L., Mason, H.L., Chalmers, R.K., Popovich, N. G. & Scott, S.A. (1996). Evaluating Pharmacy Student's Ability-Based Educational Outcomes Using an Assessment Center Approach. American Journal of Pharmaceutical Education, 60, 239-248
Rizzolo, M.A. (2012). Simulation for high-stakes evaluation: letter to the editor. Nurse Educator, 37(5), 182. doi:http://dx. doi.org/10.1097/NNE.0b013e318262eb43
Rizzolo, M.A., Kardong-Edgren, S., Oermann, M.H. & Jeffries, P.R. (2015). The National League for Nursing Project to Explore the Use of Simulation for High-Stakes Assessment: Process, Outcomes, and Recommendations. Nursing Education Perspectives, 36(5), 299-303
Rushforth, H. (2007). Objective structured clinical examination (OSCE): Review of literature and implications for nursing education. Nurse Education Today, 27, 481-490
Ryall, T., Judd, B. & Gordon, C. (2016). Simulation-based assessments in health professional education: a systematic review. Journal of Multidisciplinary Healthcare, 9, 69-82
Scalese, R., Obeso, V. & Issenberg, S.B. (2007). Simulation Technology for Skills Training and Competency Assessment in Medical Education. Journal of General Internal Medicine, 23, 46-49
Seybert, A. (2011). Patient Simulation in Pharmacy Education. American Journal of Pharmaceutical Education, 75(9), Art.187
Seybert, A.L., Kobulinsky, L.R. & McKaveney, T.P. (2008). Human patient simulation in a pharmacotherapy course. American Journal of Pharmaceutical Education, 72(2), 37
Shirwaikar, A. (2015). Objective structured clinical examination (OSCE) in pharmacy education - a trend. Pharmacy Practice, 13(4), 627
Smith, V., Muldoon, K. & Biesty, L. (2012). The Objective Structured Clinical Examination (OSCE) as a strategy for assessing clinical competence in midwifery education in Ireland: A critical review. Nurse Education in Practice, 12, 242-247
Smithburger, P.L., Kane-Gill, S.L., Ruby, C.M. & Seybert, A.L. (2012). Comparing effectiveness of 3 learning strategies: simulation-based learning, problem-based learning, and standardized patients. Simulation in Healthcare: The Journal of The Society for Medical Simulation, 7(3), 141-146. doi:http://dx.doi.org/10.1097/SIH.0b013e31823ee24d
Tavares, W., LeBlanc, V.R., Mausz, J., Sun, V. & Eva, K.W. (2014). Simulation-based assessment of paramedics and performance in real clinical contexts. Prehospital Emergency Care, 18(1), 116-122. doi:http://dx.doi.org/10.3109/ 10903127.2013.818178
van der Vleuten, C.P., Schuwirth, L., Scheele, F., Driessen, E. & Hodges, B. (2010). The assessment of professional competence: Building blocks for theory development. Best Practice and Research. Clinical Obstetrics and Gynaecology, 24, 703-719
van der Vleuten, C.P. & Schuwirth, L. W. (2005). Assessing professional competence: from methods to programmes. Medical Education, 39, 309-317
Vyas, D., Bhutada, N.S. & Feng, X. (2012). Patient simulation to demonstrate students' competency in core domain abilities prior to beginning advanced pharmacy practice experiences. American Journal of Pharmaceutical Education, 76(9), 176. doi:http://dx.doi.org/10.5688/ajpe 769176
Vyas, D., Bray, B. & Wilson, M. (2013). Use of Simulation-Based Teaching Methodologies in US Colleges and Schools of Pharmacy. American Journal of Pharmaceutical Education, 77(3), 1-5, Article 53
Vyas, D., Wombwell, E., Russell, E. & Caligiuri, F. (2010). High-fidelity patient simulation series to supplement introductory pharmacy practice experiences. American Journal of Pharmaceutical Education, 74(9), 169
WHO [World Health Organisation]. (2012). Ensuring good dispensing practice In Management Sciences for Health. MDS-3: Managing Access to Medicines and Health Technologies (ed P. Spivey), Arlington, USA.
