Difficulty and discrimination indices as quality assurance tools for assessments in a South African problem-based pharmacy programme
Keywords:
Item analysis, difficulty, discrimination, assessment, test construction, evaluation, item bankAbstract
This study investigated the difficulty and discrimination ability of examination questions in an undergraduate pharmacy programme presented at the Medunsa Campus of the University of Limpopo, South Africa. This investigation was part of an education study, which evaluated the quality of knowledge assessments in this outcomes and problem-based BPharm programme. Indices of difficulty (where a higher index characterises an easier item) and indices of discrimination were calculated for each True/False item and constructed response question from a total of 15 summative examinations in the first-to fourth-year level. We adapted the item analysis calculation methods to additionally accommodate questions counting more than one mark. Mean difficulty indices were 66.7% for True/False items without negative marking; 55.1% with negative marking, and 60.1% for constructed response questions. Discrimination indices for True/False items were 0.22 without negative marking, 0.24 with negative marking, and 0.28 for constructed response questions. Factors are discussed which potentially influence the difficulty and discrimination of examination items and the usefulness of item analysis techniques to review and improve future assessments in this programme.References
Aiken, L. R., (1982). Psychological testing and assessment. 4th Ed. Boston, MA: Allyn and Bacon.
Anderson, H. M., Anaya, G., Bird, E. & Moore, D. L. (2005). A review of educational assessment. American Journal of Pharmaceutical Education, 69(1), 84-100.
Bloom, B. S. (Ed.), Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). Taxonomy of educational objectives: the classification of educational goals. Handbook I, The cognitive domain. London: Longman Group.
Carneson, J., Delpierre, G. & Masters, K. (1996). UCT's Designing and managing multiple choice questions. University of Cape Town, South Africa. Online available from: http://web.uct.ac.za/projects/cbe/mcqman/mcqman01.html. Accessed January 31, 2001.
Dent, J. A. & Harden, R. M. (2001). A practical guide for medical teachers. Edinburgh: Churchill Livingstone.
Downing, S. M. (2002). Assessment of knowledge with written test forms. In G. R. Norman, C. P. M. van der Vleuten, & D. I. Newble (Eds.), International handbook of research in medical education, Part Two, Vol. 7 (pp. 647-672).
Dordrecht: Kluver Academic Publishers. Dreckmeyer, M. & Fraser, W. J. (1991). Classroom testing in biology and physical science. Pretoria: HAUM.
Ebel, R. L. & Frisbie, D. A. (1991). Essentials of educational measurement, 5thEdn. Englewood Cliffs: Prentice-Hall.
Fourie, S. (2004). Knowledge assessment in the MEDUNSA BPharm degree programme. [Doctoral thesis]. Medunsa: Medical University of Southern Africa.
Interim Pharmacy Council of South Africa. (1998). Draft unit standards for future pharmacists at entry level, Pretoria: South African Pharmacy Council.
Kehoe, J. (1995). Basic item analysis for multiple-choice tests. Practical Assessment, Research & Evaluation, 4(10). ERIC/AE Digest. ERIC Document Reproduction Service ED398237. Online available from: http://PAREonline.net/getvn.asp?v=4&n=10. Accessed March 12, 2003.
Linn, R. L. & Gronlund, N. E. (2000). Measurement and assessment in teaching. 8thEdn. Upper Saddle River, NJ: Prentice-Hall.
Mabope, L. A. & Meyer, J. C. (2009). Ten year‟s experience: access to and throughput of the BPharm programme at the University of Limpopo, Medunsa Campus. Paper presented at the Medunsa Academic Day, 13 August 2009. Medunsa.
McCown, R.R., Driscoll, M., & Roop, P. G. (1996). Educational psychology: a learning-centred approach to classroom practice. 2nd Edn. Boston, MA: Allyn & Bacon. Professional Testing. (2008). Building high quality examination programmes.
Online available from: http://www.proftesting.com/test_topics/steps.phpAccessed January 14, 2010.
Rudner, L.M. (1998). Item Banking. Practical Assessment, Research & Evaluation 6(4). Online available from ERIC Clearinghouse on Assessment and Evaluation: http://ericae.net/pare/getvn.asp?v=6&n=4.
Accessed August 2, 2002 and http://PAREonline.net/getvn.asp?v=6&n=4accessed February 10, 2010.
Sim, D. S-M. & Rasiah, R. I. (2006). Relationship between item difficulty and discrimination indices in True/False-type multiple choice questions of a para-clinical multidisciplinary paper. Annals of the Academy of Medicine, Singapore, 35:67-71.
Singh, T., Singh, D., & Vinod, K. P. (2003). Better medical education: principles of medical education. Indian Acadamy of Pediatrics. Online available from: http://www.edu4med.com/princpl/index.html. Accessed June 24, 2003.
Streiner, D. L. & Norman, G. R. (1995). Health measurement scales: a practical guide to their development and use. 2nd Edn. Oxford: Oxford University Press.
Summers, R., Haavik, C., Summers, B., Moola, F., Lowes, M., & Enslin, G. (2001). Pharmaceutical education in the South African multicultural society, American Journal of Pharmaceutical Education, 65(2), 150-154.
