Assessing ChatGPT’s impact on curriculum outcomes and Entrustable Professional Activities (EPAs) during Advanced Pharmacy Practice Experiences (APPEs)
DOI:
https://doi.org/10.46542/pe.2024.241.608613Keywords:
Artificial intelligence, Pharmacy education, Pharmacy student learning, Student perceptionAbstract
Background: Chat Generative Pre-trained Transformer (ChatGPT) use in higher education has been controversial. The purpose of this study was to evaluate the impact of ChatGPT on pharmacy student achievement of educational outcomes and entrustable professional activities, as outlined by the American Association of Colleges of Pharmacy (AACP), during Advanced Pharmacy Practice Experiences (APPEs).
Methods: A 20-question electronic survey was distributed to pharmacy students in the Class of 2023 and 2024. The survey evaluated the impact of ChatGPT on educational outcomes and student perceptions of the technology. Results were analysed using descriptive statistics.
Results: A total of 69 students participated in the survey. Among them, 44% (17/39) used ChatGPT during their APPE rotations. Of those, 82% (14/17) of students agreed or strongly agreed that ChatGPT helped them find, analyse, and integrate foundational knowledge of medications. Similarly, 82% (14/17) agreed or strongly agreed that ChatGPT positively impacted their approach to work and research habits. Additionally, 89% (15/17) agreed or strongly agreed that they will use ChatGPT on future APPEs and in professional practice.
Conclusion: The use of ChatGPT by pharmacy students during their APPE rotations demonstrates a potential benefit in student achievement of educational outcomes across all three domains during APPEs.
References
Abu Hammour, K., Alhamad, H., Al-Ashwal, F. Y., Halboup, A., Abu Farha, R., & Abu Hammour, A. (2023). ChatGPT in pharmacy practice: a cross-sectional exploration of Jordanian pharmacists' perception, practice, and concerns. Journal of Pharmaceutical Policy and Practice, 16(1), 115. https://doi.org/10.1186/s40545-023-00624-2
Cascella, M., Montomoli, J., Bellini, V., & Bignami, E. (2023). Evaluating the feasibility of ChatGPT in healthcare: An analysis of multiple clinical and research scenarios. Journal of Medical Systems, 47(1), 33. https://doi.org/10.1007/s10916-023-01925-4
Gilson, A., Safranek, C. W., Huang, T., Socrates, V., Chi, L., Taylor, R. A., & Chartash, D. (2023). How does ChatGPT perform on the United States Medical Licensing Examination (USMLE)? The implications of large language models for medical education and knowledge assessment. JMIR Medical Education, 9, e45312. https://doi.org/10.2196/45312
Jackson, A. (2023). These are the top 10 most popular AI tools of 2023, and how to use them to make more money. CNBC make it. https://www.cnbc.com/2023/12/24/the-top-10-ai-tools-of-2023-and-how-to-use-them-to-make-more-money.html
Khan, R. A., Jawaid, M., Khan, A. R., & Sajjad, M. (2023). ChatGPT - Reshaping medical education and clinical management. Pakistan Journal of Medical Sciences, 39(2), 605–607. https://doi.org/10.12669/pjms.39.2.7653
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., & Tseng, V. (2023). Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digital Health, 2(2), e0000198. https://doi.org/10.1371/journal.pdig.0000198
Liu, P. R., Lu, L., Zhang, J. Y., Huo, T. T., Liu, S. X., & Ye, Z. W. (2021). Application of artificial intelligence in medicine: An overview. Current Medical Science, 41(6), 1105–1115. https://doi.org/10.1007/s11596-021-2474-3
Liu, S., Wright, A. P., Patterson, B. L., Wanderer, J. P., Turer, R. W., Nelson, S. D., McCoy, A. B., Sittig, D. F., & Wright, A. (2023). Assessing the value of ChatGPT for clinical decision support optimization. medRxiv: the preprint server for health sciences, 2023.02.21.23286254. https://doi.org/10.1101/2023.02.21.23286254
Medina, M. S., Farland, M. Z., Conry, J. M., Culhane, N., Kennedy, D. R., Lockman, K., Malcom, D. R., Mirzaian, E., Vyas, D., Stolte, S., Steinkopf, M., & Ragucci, K. R. (2023). The AACP Academic Affairs Committee's Final 2022 Curricular Outcomes and Entrustable Professional Activities (COEPA) for pharmacy graduates to replace 2013 CAPE and 2016 EPAs. American Journal of Pharmaceutical Education, 87(8), 100558. https://doi.org/10.1016/j.ajpe.2023.100558
OpenAI. (2024). ChatGPT pricing. Retrieved February 8, 2024, from https://openai.com/chatgpt/pricing
Patel, C.R., Pandya, S.K., & Sojitra, B.M. (2023). Perspectives of ChatGPT in pharmacology education, and research in health care: A narrative review. Journal of Pharmacology and Pharmacotherapeutics, 14(3), 171–177. https://doi.org/10.1177/0976500X231210427
Raza, M. A., Aziz, S., Noreen, M., Saeed, A., Anjum, I., Ahmed, M., & Raza, S. M. (2022). Artificial Intelligence (AI) in pharmacy: An overview of innovations. Innovations in Pharmacy, 13(2), 4839. https://doi.org/10.24926/iip.v13i2.4839
Roosan, D., Padua, P., Khan, R., Khan, H., Verzosa, C., & Wu, Y. (2024). Effectiveness of ChatGPT in clinical pharmacy and the role of artificial intelligence in medication therapy management. Journal of the American Pharmacists Association: JAPhA, 64(2), 422–428.e8. https://doi.org/10.1016/j.japh.2023.11.023
Schutz, N., Olsen, C. A., McLaughlin, A. J., Yi, W. M., Nelson, S. D., Kalichira, A. L., Smith, A. H., Miller, K. A., Le, T., Chaffee, B. W., Worthy Woodbury, C. D. R. K., & Patel, H. (2020). ASHP statement on the use of artificial intelligence in pharmacy. American Journal of Health-System Pharmacy: AJHP, 77(23), 2015–2018. https://doi.org/10.1093/ajhp/zxaa249
