Assessment of Pharmacists’ Patient Care Competencies: Validity Evidence from Ontario (Canada)’s Quality Assurance and Peer Review Process
Keywords:
Direct assessment, Maintenance of competence, Competency assessment, Pharmacy educationAbstract
Maintenance of competence is integral to health care practitioners’ continuing professional development. The adequacy and value of indirect assessment of competence (through, for example, learning portfolios or attendance at educational events) has been questioned. Direct assessment (such as written tests of clinical knowledge or objective structured clinical examinations, OSCEs) has been advocated as a more meaningful indicator of a practitioner’s competence. This paper describes the Ontario (Canada) College of Pharmacists’ experience with direct assessment through the Peer Review/Quality Assurance process. This process consists of a self- assessment questionnaire, ongoing maintenance of a learning portfolio, a written test of clinical knowledge, and an OSCE. Each year, a randomly selected group of pharmacists in Ontario undertake the Peer Review process. After five years of operation, 992 pharmacists had participated in this program; 86% of participants met or exceeded standards and were encouraged to continue with their own professional development while 14% of participants did not meet standards in identified assessment areas, and were directed to a peer-assisted process to facilitate professional development. Findings suggest individuals who were educated outside Canada or the United States, those in community pharmacy practice, and those who had been in practice 25 years or more demonstrated greatest difficulty in meeting standards. The implications of these results for pharmacy practice and professional development are discussed as are issues related to direct and indirect assessment of clinical skills.References
Austin, Z., O’Byrne, C., Pugsley, J. and Queroz, L. (2003a) “Development and validation of an objective structured clinical examination (OSCE) for entry-to-practice certification: the Canadian experience”, American Journal of Pharmaceutical Education 62(3), 76–80.
Austin, Z., Croteau, D., Marini, A. and Violato, C. (2003b) “Continuous professional development: The Ontario experi- ence in professional self-regulation through quality assurance and peer review”, American Journal of Pharmaceutical Education 62(2), 56–63.
Bashook, P. and Parboosingh, J. (1998) “Recertification and the maintenance of competence”, British Medical Journal 316, 545 – 548.
Benson, J. (1991) “Certification and recertification: one approach to professional accountability”, Annals of Internal Medicine 114, 228 – 232.
Campbell, C. (1996) “The maintenance of competence pro- gramme of the Royal College of Physicians and Surgeons of Canada”, Postgraduate Medical Journal 72(Suppl. 1), s41–s42.
Cannon, C. and Waters, L. (1993) “Preparing for mandatory continuing education–assessing interests”, Journal of Continu- ing Education in Nursing 24(4), 148–152.
Cunnington, J. and Norman, G. (2000) “Certification and re-certification: are they the same?”, Academic Medicine 75(6), 617 – 619.
Davis, P., Russell, A. and Skeith, K. (1997) “The use of standardized patients in the performance of a needs assessment and development of a CME intervention in rheumatology for primary care physicians”, Journal of Rheumatology 24(10), 1995–1999.
Davis, D., Thomson O’Brien, M., Freemantle, N., Mazmanian, P. and Taylor-Vaisey, A. (1999) “Impact of formal continuing medical education: do conferences, workshops, rounds and other traditional forms of continuing education activities change physician behaviour or health outcome?”, Journal of the American Medical Association 282, 867–874.
Dunn, E., Bass, M., Williams, J., Borgiel, A., Macdonald, P. and Spassoff, R. (1988) “Study of relation of continuing medical education to quality of family physicians’ care”, Journal of Medical Education 63, 775–784.
Ebel, R.L. and Frisbe, D.A. (1986) Essentials of Educational Measurement (Prentice Hall, Toronto).
Fielding, D., Page, G., Fevang, L. and Thomas, N. (1981) “Competency assessment: a progress report on British Columbia’s program”, American Journal of Pharmaceutical Education 45, 178–183.
Fielding, D., Page, G., Schulzer, M., Rogers, W. and O’Byrne, C. (1992) “Assuring continuing competency: identification and validation of a practice-based assessment blueprint”, American Journal of Pharmaceutical Education 56, 21–29.
Fielding, D., Rogers, W., Tench, E., O’Byrne, C., Page, G. and Schulzer, M. (2001) “Predictors of continuing competence”, American Journal of Pharmaceutical Education 65, 107–118.
Hambleton, R.K. (1995) Setting standards in licensure tests Paper presented at 103rd Convention of the American Psychological Association, New York, NY.
Kremer, B. (1991) “Physician recertification and outcomes assessment”, Evaluation and the Health Professions 14(2), 187 – 200.
LaDuca, A., Leone-Perkins, M. and de Champlain, A. (1997) “Evaluating continuing competence of physicians through multiple assessment modalities: the physicians’ continued competence assessment program”, Academic Medicine 72(5), 457 – 458.
Lim, T., Soraya, A., Ding, L. and Morad, Z. (2002) “Assessing doctors’ competence: application of CUSUM technique in monitoring doctors’ performance”, International Journal for Quality in Health Care 14(3), 251–258.
Manning, P.R. (1987) “The past, present and future of continuing medical education”, Journal of the American Medical Association 258, 3524–3536.
Norcini, J. (1993) “Recertification in the medical specialties”, Academic Medicine 69(10 Suppl.), s90–s94.
Norcini, J., Lipner, R. and Downing, S. (1996) “How meaningful are scores on a take-home recertification examination?”, Academic Medicine 71(10 Suppl.), s71–s73.
Page, G., Bates, J., Dyer, S., Vincent, D., Bordage, G., Jacques, A., Sindon, A., Kaigas, T. and Norman, G. (1995) “Physician assessment and physician enhancement programs in Canada”, Canadian Medical Association Journal 153(12), 1723 – 1728.
Parboosingh, J. and Gondocz, S. (1993) “The maintenance of competence (MOCOMP) program: motivating specialists to appraise the quality of their continuing education activities”, Canadian Journal of Surgery 36(1), 29–32.
