Evaluation of differences in postgraduate year one pharmacy residency application rubrics
DOI:
https://doi.org/10.46542/pe.2026.261.127136Keywords:
Application, Candidate, Postgraduate training, Residency, RubricAbstract
Background: The American Society of Health-System Pharmacists standards require pharmacy residency programmes to systematically evaluate candidates. This study aimed to assess differences among eight PGY1 pharmacy residency application rubrics used to select candidates for interviews.
Methods: In December 2018, fifty applicants to the University of Kentucky HealthCare’s PGY1 Pharmacy Residency Programme were evaluated using eight different rubrics. A subgroup of sixteen applicants had their scores assessed by multiple scorers to analyse inter-rater variability. The main goal was to examine score variability across all applicants. Scores were normalised to a common scale, and statistical analysis included descriptive statistics and ANOVA tests.
Results: The evaluation of a candidate pool (n = 520) using eight unique rubrics yielded an overall mean score of 66.98 out of 100, with significant score differences across rubrics (p < 0.001). Inter-rater variability was low, with a maximum difference of six percent in scores. All rubrics assessed work experience, leadership positions, and publications, but not all included letters of recommendation. Letters of recommendation, work experience, and rotation experiences received the highest weight in the rubrics.
Conclusion: There was notable variability in scores among eight different PGY1 pharmacy residency application rubrics. While aspects like rotation experiences were highly ranked, others, such as letters of recommendation, varied significantly. Programmes prioritise different elements based on their preferences, leading to differences in applicant evaluations. This allows programmes to find candidates that fit their practice, but applicants should be aware that their portfolios may be assessed differently across programmes.
References
ASHP. (2023). ASHP accreditation standards for postgraduate pharmacy residency programs. Retrieved September 7 from https://www.ashp.org/-/media/assets/professional-development/residencies/docs/examples/ASHP-Accreditation-Standard-for-Postgraduate-Residency-Programs-effective-July-2023.pdf
Atyia, S. A., Paloucek, F. P., Butts, A. R., Oyler, D. R., Martin, C. A., Schadler, A. D., & Cook, A. M. (2020). Impact of PhORCAS references on overall application score for postgraduate year 1 pharmacy residency candidates. American Journal of Health-System Pharmacy, 77(15), 1237‒1242. https://doi.org/10.1093/ajhp/zxaa152
Bewick, V., Cheek, L., & Ball, J. (2004). Statistics review 9: One-way analysis of variance. Critical Care (London, England), 8(2), 130‒136. https://doi.org/10.1186/cc2836
Blake, E. W., Friesner, D., Gettig, J. P., Hajjar, E., Gentry, E. J., & Kline, J. M. (2015). Comparing pharmacy practice faculty perceptions of first-year post-graduate residency (PGY1) selection criteria to those reported by PGY1 residency directors. Currents in Pharmacy Teaching and Learning, 7, 20‒28.
Cho, J. C. (2018). Selecting candidates for pharmacy residencies: A national survey of residency programme directors. Journal of Clinical Pharmacy and Therapeutics, 43(6), 844‒848. https://doi.org/10.1111/jcpt.12723
Crane, J. T., & Ferraro, C. M. (2000). Selection criteria for emergency medicine residency applicants. Academic Emergency Medicine, 7(1), 54‒60. https://doi.org/10.1111/j.1553-2712.2000.tb01892.x
Cullen, M. W., Reed, D. A., Halvorsen, A. J., Wittich, C. M., Kreuziger, L. M., Keddis, M. T., McDonald, F. S., & Beckman, T. J. (2011). Selection criteria for internal medicine residency applicants and professionalism ratings during internship. Mayo Clinic Proceedings, 86(3), 197‒202. https://doi.org/10.4065/mcp.2010.0655
DeLisa, J. A., Jain, S. S., & Campagnolo, D. I. (1994). Factors used by physical medicine and rehabilitation residency training directors to select their residents. American Journal of Physical Medicine & Rehabilitation, 73(3), 152‒156. https://doi.org/10.1097/00002060-199406000-00002
Ensor, C. R., Walker, C. L., Rider, S. K., Clemente, E. U., Ashby, D. M., & Shermock, K. M. (2013). Streamlining the process for initial review of pharmacy residency applications: an analytic approach. American Journal of Health-System Pharmacy, 70(19), 1670‒1675. https://doi.org/10.2146/ajhp120769
Gohlke, A. L., Ray, D. B., El-Ibiary, S. Y., & Barletta, J. F. (2014). Characteristics of the ideal postgraduate year 1 pharmacy practice residency candidate: A survey of residency programme directors. Journal of Pharmacy Practice, 27(1), 84‒88. https://doi.org/10.1177/0897190013504962
Harris, P. A., Taylor, R., Thielke, R., Payne, J., Gonzalez, N., & Conde, J. G. (2009). Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. Journal of Biomedical Informatics, 42(2), 377‒381. https://doi.org/10.1016/j.jbi.2008.08.010
Hillebrand, K., Leinum, C. J., Desai, S., Pettit, N. N., & Fuller, P. D. (2015). Residency application screening tools: A survey of academic medical centers. American Journal of Health-System Pharmacy, 72(11 Suppl 1), S16‒19. https://doi.org/10.2146/ajhp150093
Hu, D., Veenhouwer, D., McCoy, J., & Caselnova, D. A., 3rd. (2019). A screening tool to identify qualified pharmacy residency candidates. American Journal of Pharmaceutical Education, 83(4), 6714. https://doi.org/10.5688/ajpe6714
Jellinek-Cohen, S. P., Cohen, V., Bucher, K. L., & Likourezos, A. (2012). Factors used by pharmacy residency programmes to select residents. American Journal of Health-System Pharmacy, 69(13), 1105‒1106, 1108. https://doi.org/10.2146/ajhp100615
Macias-Moriarity, L. Z., Fetterman, J. W., Thomas, M. C., & Stajich, G. V. (2015). Residency programme director’s perception of PGY-1 applicants: Results of the National 35½ Second Survey. Currents in Pharmacy Teaching and Learning, 7(4), 465‒469. https://doi.org/https://doi.org/10.1016/j.cptl.2015.04.009
National Matching Services, I. (2024, March 2024). ASHP match statistics. Retrieved March 26 from https://natmatch.com/ashprmp/stats.html
Pate, A. N., Mills, A. R., Fleming, J. W., Phan, H. K., Street, M., & Pitcock, J. J. (2023). Residency application content and considerations based on residency director review of a fictitious CV: What really matters? American Journal of Health-System Pharmacy, 80(17), 1147‒1156. https://doi.org/10.1093/ajhp/zxad068
Poirier, M. P., & Pruitt, C. W. (2003). Factors used by pediatric emergency medicine programme directors to select their fellows. Pediatric Emergency Care, 19(3), 157‒161. https://doi.org/10.1097/01.pec.0000081236.98249.ed
Reed, B. N. (2021). Rethinking letters of reference: Narrowing the gap between evidence and practice. American Journal of Health-System Pharmacy, 78(23), 2175‒2179. https://doi.org/10.1093/ajhp/zxab243
Reed, B. N., Noel, Z. R., Heil, E. L., Shipper, A. G., & Gardner, A. K. (2021). Surveying the selection landscape: A systematic review of processes for selecting postgraduate year 1 pharmacy residents and key implications. Journal of the American College of Clinical Pharmacy, 4(2), 245‒256. https://doi.org/10.1002/jac5.1334
Skrupky, L. P., Dy-Boarman, E. A., Gurgle, H. E., Isaacs, A. N., McCreary, E. K., Nisly, S. A., Paloucek, F., Peterson, D. M., Smith, A., Schramm, G. E., Education, T., Practice, T., & Network, R. (2021). Letters of reference for PGY1 pharmacy residency candidates: A survey of residency programme directors and opinion statement. Journal of the American College of Clinical Pharmacy, 4(3), 379‒389. https://doi.org/https://doi.org/10.1002/jac5.1370
Warfield, M. N., Nei, S. D., Cole, K. C., & Krauter, A. K. (2024). Assessment of postgraduate Year 1 residency programme director preferences and priorities in evaluation of pharmacy residency candidates. American Journal of Pharmaceutical Education, 88(12), 101323. https://doi.org/10.1016/j.ajpe.2024.101323
