article/article.tpl

Integrated simulation-based skills assessment for evaluating pharmacist competence: A scoping review

Hayley Alison Croft, Beverley Glass, Conor Gilligan, Rohan Rasiah, Tracy Levett-Jones

Abstract

Background: Integrated assessment of health professional competence more closely represents performance of tasks in clinical practice, than assessment models that evaluate fragmented competencies. Simulation continues to expand the range of assessment methods available for training healthcare professionals, however, there is limited understanding of the potential use of integrated simulation-based assessments in pharmacy education.

Objective: To review the range of integrated simulation-based assessments used across the healthcare professions, and explore their applicability to pharmacy education.

Methods: Medline, International Pharmaceutical Abstracts (IPA), EMBASE, CINAHL, ERIC and Scopus were searched for articles in English published between January 1975 and May 2019, to identify studies that reported on the use of integrated simulation-based assessments in medicine, nursing, allied health and pharmacy.

Results: Searches identified 2, 440 records for screening; 59 full-text articles were screened for eligibility, with a total of 20 studies meeting the inclusion criteria for the review. The findings show evidence for a variety of simulation approaches to enable the evaluation of integrated competencies, however the validity and reliability of such approaches is less well documented. Most studies have been conducted in medicine, nursing and allied health with only two studies conducted in pharmacy. All studies used multidimensional assessment instruments but only fourteen were tested for reliability and validity.

Conclusion: Despite the limited number of published studies specifically pertaining to integrated simulation-based assessment in pharmacy, the current approach in other healthcare disciplines provides support for future application in pharmacy.


Keywords

Integrated Skills Assessment; Simulation; Health Professionals; Pharmacy


Full Text:

PDF

References

Aggarwal, R., Mytton, O., Derbrew, M., Hananel, D., Heydenberg, M., Barry Issenberg, S., MacAulay, C., Mancini, M.E., Morimoto, T., Soper, N., Ziv, A. & Reznick, R. (2010). Training and simulation for patient safety. Journal of Quality and Safety in Health Care, 19(Suppl. 2), i34-i43

Auffermann, W., Henry, T., Little, B., Tigges, S. & Tridandapani, S. (2015). Simulation for Teaching and Assessment of Nodule Perception on Chest Radiography in Nonradiology Health Care Trainees. Journal of the American College of Radiology, 12(11)

Australian Pharmacy Council. (2017). Intern Year Blueprint Literature Review Canberra, ACT: Australian Pharmacy Council Limited

Barman, A. (2005). Critiques on the objective structured clinical examination. Annals of the Academy of Medicine, 34, 478-482

Benedict, N., Smithburger, P., Calabrese, A., Empey, P., Kobulinsky, L., Seybert, A., Waters, T., Drab, S., Lutz, J., Farkas, D. & Meyer, S. (2017). Blended Simulation Progress Testing for Assessment of Practice Readiness. American Journal of Pharmaceutical Education, 81(1), 1-13 Article 14

Boulet, J. (2008). Summative Assessment in Medicine: The Promise of Simulation for High-stakes Evaluation. Academic Emergency Medicine, 15, 1017-1024. doi:10.1111/j.1553- 2712.2008.00228.x

Boulet, J., Murray, D., Kras, J., Woodhouse, J., McAllister, J. & Ziv, A. (2003). Relaibility and validity of a simulation-based acute care skills assessment for medical students and residents. Anesthesiology, 99(6), 1270-1280

Boulet, J.R. (2008). Summative assessment in medicine: the promise of simulation for high-stakes evaluation. Academic Emergency Medicine, 15(11), 1017-1024. doi:http://dx.doi.org/10.1111/j.1553-2712.2008.00228.x

Boulet, J.R., Murray, D., Kras, J., Woodhouse, J., McAllister, J. & Ziv, A. (2003). Reliability and validity of a simulation-based acute care skills assessment for medical students and residents. Anesthesiology, 99(6), 1270-1280

Bray, B.S., Schwartz, C.R., Odegard, P.S., Hammer, D.P. & Seybert, A.L. (2011). Assessment of human patient simulation-based learning. American Journal of Pharmaceutical Education, 75(10), 208. doi:http://dx.doi.org/10.5688/ajpe7510208

Bujack, L., McMillan, M., Dwyer, J. & Hazelton, M. (1991). Assessing comprehensive nursing performance: the Objective Structured Clinical Assessment (OSCA). Part 2 - Report of the evaluation project. Nurse Education Today, 11, 248-255

DeMaria, S.Jr., Samuelson, S.T., Schwartz, A.D., Sim, A.J. & Levine, A.I. (2013). Simulation-based assessment and retraining for the anesthesiologist seeking reentry to clinical practice: a case series. Anesthesiology, 119(1), 206-217. doi:http://dx.doi.org/10.1097/ALN.0b013e31829761c8

Douglas, H., Raban, M., Walter, S. & Westbrook, J. (2017). Improving our understanding of multi-tasking in healthcare: Drawing together the cognitive psychology and healthcare literature. Applied Ergonomics, 59, 45-55

East, L., Peters, K., Halcomb, E., Raymond, D. & Salamonson, Y. (2014). Evaluating Objective Structured Clinical Assessment (OSCA) in undergraduate nursing. Nurse Education in Practice, 14, 461-467

Epstein, R.M. (2007). Assessment in medical education. New England Journal of Medicine, 356(4), 387-396

Epstein, R.M. & Hundert, E.M. (2002). Defining and assessing professional competence. Journal of the American Medical Association, 287, 226-235

Gimpel, J., Boulet, D. & Errichetti, A. (2003). Evaluating the clinical skills of osteopathic medical students. Journal of Americal Osteopathy Association, 103(6), 267-279

Gupta, P., Dewan, P. & Singh, T. (2010). Objective Structured Clinical Examination (OSCE) Revisited. Medical Education, 47, 911-919

Hirsch, A.C. & Parihar, H.S. (2014). A capstone course with a comprehensive and integrated review of the pharmacy curriculum and student assessment as a preparation for advanced pharmacy practice experiences. American Journal of

Pharmaceutical Education, 78(10), 192. doi:http://dx.doi.

org/10.5688/ajpe7810192

Ilgen, J., Ma, I.W.Y., Hatala, R. & Cook, D.A. (2015). A systematic review of validity evidence for checklists versus global rating scales in simulation-based assessment. Medical Education, 49(2), 161-173

Isenberg, G.A., Berg, K.W., Veloski, J.A., Berg, D.D., Veloski, J.J. & Yeo, C.J. (2011). Evaluation of the use of patient-focused simulation for student assessment in a surgery clerkship. American Journal of Surgery, 201(6), 835-840. doi:http://dx.doi.org/10.1016/j.amjsurg.2010.01.034

Judd, B., Scanlan, J., Alison, J., Waters, D. & Gordon, C. (2016). The validity of a professional competence tool for physiotherapy students in simulation-based clinical education: a Rasch analysis. BMC Medical Education, 16(196), 1-10

Kesten, K.S., Brown, H.F. & Meeker, M.C. (2015). Assessment of APRN Student Competency Using Simulation: A Pilot Study. Nursing Education Perspectives, 36(5), 332-334

Khan, K., Ramachandran, S., Gaunt, K. & Pushkar, P. (2013). The Objective Structured Clinical Examination (OSCE): AMEE Guide No. 81. Part I: An historical and theoretical perspective. Medical Teacher, 35, e1437-1446

Kneebone, R., Nestel, D., Yadollahi, F., Brown, R., Nolan, C., Durack, J., Brenton, H., Moulton, C., Archer, J. & Darzi, A. (2006). Assessing procedural skills in context: Exploring the feasibility of an Integrated Procedural Performance Instrument (IPPI). Medical Education, 40(11), 1105-1114

Kreiter, C.D., Haugen, T., Leaven, T., Goerdt, C., Rosenthal, N., McGaghie, W.C. & Dee, F. (2011). A report on the piloting of a novel computer-based medical case simulation for teaching and formative assessment of diagnostic laboratory testing. Medical Education Online, 16.

doi:http://dx.doi.org/10.3402/meo.v16i0.5646

Lammers, R., Byrwa, M., Fales, W. & Hale, R. (2009). Simulation-based assessment of paramedic pediatric resuscitation skills. Journal of Prehospital Emergency Care, 13(3), 345-356

LeBlanc, V.R., Tabak, D., Kneebone, R., Nestel, D., MacRae, H. & Moulton, C.-A. (2009). Psychometric properties of an integrated assessment of technical and communication skills. The American Journal of Surgery, 197, 96-101

Lipner, R., Messenger, J., Kangilaski, R., Baim, D., Holmes, D., Williams, D. & King, S. (2010). A technical and cognitive skills eveluation of performance in interventional cardioogy procedures using medical simulation. Journal for Simulation in Healthcare, 5(2), 65-74

McBride, M., Waldrop, W., Fehr, J., Boulet, J. & Murray, D. (2011). Simulation in pediatrics: the reliability and validity of a multiscenario assessment. Pediatrics, 128(2), 335-343

McGaghie, W.C., Issenberg, S.B., Petrusa, E.R. & Scalese, R.J. (2010). A critical review of simulation-based medical education research: 2003-2009. Medical Education, 44(1), 50-63. doi:http://dx.doi.org/10.1111/j.1365-2923.2009. 03547.x

Medina, M., Plaza, C., Stowe, C., Robinson, E., DeLander, G., Beck, D., Melchert, R.B., Supernaw, R.B., Roche, V.F., Gleason, B.L., Strong, M.N., Bain, A., Meyer, G.E., Dong, B.J., Rochon, J. & Johnston, P. (2013). Center for the Advancement of Pharmacy Education 2013 Educational Outcomes. American Journal of Pharmaceutical Education, 77(8 ), Article 162

Mikasa, A., Cicero, T. & Adamson, K. (2013). Outcome-Based Evaluation Tool to Evaluate Student Performance in High-Fidelity Simulation. Clinical Simulation in Nursing, 9(9), e361-e367

Mitchell, M., Henderson, A., Groves, M., Dalton, M. & Nulty, D. (2009). The objective structured clinical examination (OSCE): Optimising its value in the undergraduate nursing curriculum. Nurse Education Today, 29, 398-404

Moher, D., Liberati, A., Tetzlaff, J. & Altman, D. (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Annals of Internal Medicine, 151(4), 264-269

Mudumbai, S.C., Gaba, D.M., Boulet, J.R., Howard, S.K. & Davies, M.F. (2012). External validation of simulation-based assessments with other performance measures of third-year anesthesiology residents. Simulation in Healthcare: The Journal of The Society for Medical Simulation, 7(2), 73-80. doi:http://dx.doi.org/10.1097/SIH.0b013e31823d018a

Panzarella, K. & Manyon, A. (2007). A Model for Integrated Assessment of Clinical Competence. Journal of Allied Health, 36(3), 157-164

Pharmaceutical Society of Australia. (2017). Dispensing Practice Guidelines Canberra, ACT: Pharmaceutical Society of Australia Ltd.

Pittenger, A., Chapman, S., Frail, C., Moon, J., Undeberg, M. & Orzoff, J. (2016). Entrustable Professional Activities for Pharmacy Practice. American Journal of Pharmaceutical Education, 80(4), Article 57

Purkerson, D.L., Mason, H.L., Chalmers, R.K., Popovich, N. G. & Scott, S.A. (1996). Evaluating Pharmacy Student's Ability-Based Educational Outcomes Using an Assessment Center Approach. American Journal of Pharmaceutical Education, 60, 239-248

Rizzolo, M.A. (2012). Simulation for high-stakes evaluation: letter to the editor. Nurse Educator, 37(5), 182. doi:http://dx. doi.org/10.1097/NNE.0b013e318262eb43

Rizzolo, M.A., Kardong-Edgren, S., Oermann, M.H. & Jeffries, P.R. (2015). The National League for Nursing Project to Explore the Use of Simulation for High-Stakes Assessment: Process, Outcomes, and Recommendations. Nursing Education Perspectives, 36(5), 299-303

Rushforth, H. (2007). Objective structured clinical examination (OSCE): Review of literature and implications for nursing education. Nurse Education Today, 27, 481-490

Ryall, T., Judd, B. & Gordon, C. (2016). Simulation-based assessments in health professional education: a systematic review. Journal of Multidisciplinary Healthcare, 9, 69-82

Scalese, R., Obeso, V. & Issenberg, S.B. (2007). Simulation Technology for Skills Training and Competency Assessment in Medical Education. Journal of General Internal Medicine, 23, 46-49

Seybert, A. (2011). Patient Simulation in Pharmacy Education. American Journal of Pharmaceutical Education, 75(9), Art.187

Seybert, A.L., Kobulinsky, L.R. & McKaveney, T.P. (2008). Human patient simulation in a pharmacotherapy course. American Journal of Pharmaceutical Education, 72(2), 37

Shirwaikar, A. (2015). Objective structured clinical examination (OSCE) in pharmacy education - a trend. Pharmacy Practice, 13(4), 627

Smith, V., Muldoon, K. & Biesty, L. (2012). The Objective Structured Clinical Examination (OSCE) as a strategy for assessing clinical competence in midwifery education in Ireland: A critical review. Nurse Education in Practice, 12, 242-247

Smithburger, P.L., Kane-Gill, S.L., Ruby, C.M. & Seybert, A.L. (2012). Comparing effectiveness of 3 learning strategies: simulation-based learning, problem-based learning, and standardized patients. Simulation in Healthcare: The Journal of The Society for Medical Simulation, 7(3), 141-146. doi:http://dx.doi.org/10.1097/SIH.0b013e31823ee24d

Tavares, W., LeBlanc, V.R., Mausz, J., Sun, V. & Eva, K.W. (2014). Simulation-based assessment of paramedics and performance in real clinical contexts. Prehospital Emergency Care, 18(1), 116-122. doi:http://dx.doi.org/10.3109/ 10903127.2013.818178

van der Vleuten, C.P., Schuwirth, L., Scheele, F., Driessen, E. & Hodges, B. (2010). The assessment of professional competence: Building blocks for theory development. Best Practice and Research. Clinical Obstetrics and Gynaecology, 24, 703-719

van der Vleuten, C.P. & Schuwirth, L. W. (2005). Assessing professional competence: from methods to programmes. Medical Education, 39, 309-317

Vyas, D., Bhutada, N.S. & Feng, X. (2012). Patient simulation to demonstrate students' competency in core domain abilities prior to beginning advanced pharmacy practice experiences. American Journal of Pharmaceutical Education, 76(9), 176. doi:http://dx.doi.org/10.5688/ajpe 769176

Vyas, D., Bray, B. & Wilson, M. (2013). Use of Simulation-Based Teaching Methodologies in US Colleges and Schools of Pharmacy. American Journal of Pharmaceutical Education, 77(3), 1-5, Article 53

Vyas, D., Wombwell, E., Russell, E. & Caligiuri, F. (2010). High-fidelity patient simulation series to supplement introductory pharmacy practice experiences. American Journal of Pharmaceutical Education, 74(9), 169

WHO [World Health Organisation]. (2012). Ensuring good dispensing practice In Management Sciences for Health. MDS-3: Managing Access to Medicines and Health Technologies (ed P. Spivey), Arlington, USA.


Refbacks

  • There are currently no refbacks.
article/comments.tpl article/footer.tpl