The Relationship of Difficulty and Discrimination Indices with Physiology Teaching and Learning

By Sadaf Fatima¹, Syed Tousif Ahmed², Shazia Hashmat², Afshan Iftikhar³, Touqeer Alam Hashmi⁴

AFFILIATIONS:

1. Department of Physiology, Jinnah Medical and Dental College.
2. Department of Physiology, Ziauddin Medical College, Ziauddin University.
3. Department of Physiology, Lahore Medical and Dental college, Lahore.
4. Department of Examination, Ziauddin University, Karachi, Pakistan.

DOI:https://doi.org/10.36283/PJMD10-2/019

ABSTRACT

Background: Item analysis, a valuable process conducted after the examination, gives information regarding the reliability of scores and validity of items tested. Difficulty index (P) and Discrimination index (DI) are two parameters, which aid in analyzing the standard of Multiple Choice Questions (MCQs). The study was aimed to find the relationship of difficulty and discrimination indices with teaching and learning in Physiology.

Methods: Students (n=100) included were MBBS 1^st year from September 2017 to March 2018. The MCQs of Physiology semester II courses were analyzed in two Continuous Assessment Tests (CATs). CAT I included Respiratory: 15/50 and CAT II Hemopoietic: 14/60 and Semester Exam 40/80. The items were examined for difficulty index (p-value) and discrimination index (DI). Items having p-value between 0.3 to 0.7 and DI > 0.2 were considered as having good difficulty and discrimination indices respectively.

Results: Out of 15 items in Respiratory CAT, 73% had a mean p=0.3 -0.7 and 27% had mean p= 0.23. The mean DI value was 0.31. Out of 14 items in CAT of Hemopoietic, 85% had a p-value between 0.3 -0.7 and 15% had mean p=0.2. The mean DI value was 0.41. Out of 40 items in Semester exam, 82% items had p value b/w 0.3 – 0.7 and 18 % items had mean p=0.18. Overall mean DI value was 0.28.

Conclusion: The item analysis of majority of one best type MCQs of CATs and semester exam in Physiology had good difficulty and discrimination indices and matched teaching and learning strategy in Physiology.

Keywords: Difficulty Index; Discrimination Index; Physiology; Teaching; Learning.

Assessment drives learning because it includes both formative and summative knowledge, and has a strong influence on learning^1-2. It is considered a vital factor in leading the learners for accomplishing the goal³. The assessment of knowledge, skills and competence of undergraduate medical students has great significance because of the demand to be a safe practitioner in future³. Physiology is one of the basic science subjects taught to undergraduate medical students⁴. It is essential due to its application in clinical practice⁵. In medical education, knowledge of Physiology is assessed routinely in the pre-clinical years⁶.

It is tremendously important that assessment should be valid and adequately discriminate between satisfactory and unsatisfactory performers³. One best (OB) Multiple choice question (MCQ) is a frequently used tool to assess the cognitive competence of the student^6,7. In this type of assessment, the student selects the best possible answer from the given list⁸. MCQs are widely used because of objectivity^3,7, removal of assessor’s bias³ and thorough coverage of the subject matter in a short time^3,7. MCQs if formulated correctly, can test higher levels of cognition and differentiate between high and low achievers⁹.

Item analysis analyzes the student responses to individual test items (MCQs) to evaluate the distinctive quality of those items and test as a whole^10,11. It is a valuable process, which is carried out after examination that gives information regarding the reliability and validity of a test⁸. It allows observing the characteristics of a specific item and can be used to make sure that items are of an acceptable standard to be included in a test, needs to be reviewed or discarded from the test^12,13.

Difficulty index (P) and Discrimination index (DI) are two parameters that help in examining the standard of MCQ questions used in an examination¹², with abnormal values signifying poor quality. Difficulty index indicates the percentage of correct answers to total responses; also referred to as the p-value¹⁴. Items having a p-value between 0.3-0.7 are regarded as having a good difficulty index as it shows that items are of moderate difficulty⁷. In general, items of moderate difficulty are to be chosen to those that are much easier or much harder¹⁴. The greater the number of questions in a paper with moderate difficulty, the better is the quality of assessment. The discrimination index (DI), which is also known as the point biserial correlation, reflects discrimination between students with different levels of achievement¹². For an item to discriminate effectively between weaker and stronger candidates, the stronger candidates should perform better on the item than the weaker candidates¹². Items having DI > 0.2 were regarded as having good discrimination index⁷. Such items contribute to the examination’s overall ability to discriminate stronger from weaker candidates.

Items writing errors are the technical flaws that if present in any MCQ, they can influence student’s performance³. The MCQs are labeled as poorly developed because of item writing flaws⁷. They include ambiguous stems, negatively worded options, options that are not plausible, contains irrelevant information⁷. All of them affect the quality of the MCQs and makes the assessment process less reliable and valid⁷. It is extremely important to align teaching with assessment. The student performance in exam reflects their learning and understanding of the subject. It is not only important to teach well but it is also essential to design assessment appropriately matched with the teaching. The greater the number of questions with appropriate p value and DI, the more significant is the assessment. To emphasize the significance of item analysis and to express its association with teaching and learning, this study aimed to find out the relationship of difficulty and discrimination indices with teaching and learning in physiology.

The study design was cross sectional. The study duration was six months from September 2017 to March 2018. In this study, the secondary data was acquired from the examination department in the form of item analysis. The study population included 100 First Year MBBS students. The paper is comprised of one-best MCQs, having 5 options each. The same faculty members who taught the particular topic developed the Physiology MCQs. The physiology MCQs included in CATs of courses taught in semester II, Respiratory system and Hemopoietic system and the Physiology MCQs of the semester II exam, were analyzed. There were 15/50 physiology one-best MCQs in Respiratory system CAT (Figure 1a), 14/60 in Hemopoietic system CAT (Figure 1b) and 40/80 one-best MCQs in the semester exam (Figure 1c). The MCQs were analyzed for difficulty index (p-value) and discrimination index (DI). Items having p-value between 0.3-0.7 ⁷ and DI > 0.2 ⁷ were regarded as having good difficulty and discrimination indices respectively.

Figure 1a: Proportion of difficulty index in physiology section of respiratory system CAT.

Figure 1b: Proportion of difficulty index in physiology section of hemopoietic and immune system CAT.

Figure 1c: Proportion of difficulty index in physiology section of semester II Exam.

Descriptive statistics were used for analysis of data. Data is expressed in terms of p-value and DI. The difficulty index was calculated using the formula p=R/T where p is the item difficulty index, R is the number of correct responses, and T is the total number of responses. The discrimination index was calculated for each item using the formula DI=UG-LG/n where DI is discrimination index, UG is a number of students in the upper group who got an item correct, LG is number of students in lower group. The mean p-value and mean DI value was calculated.

Out of 15 Physiology items in CAT of Respiratory system, 73% were found to have p-value b/w 0.3 -0.7 and 27% items had mean p-value 0.23. The mean DI value was 0.31. Out of 14 Physiology questions in CAT of Hemopoietic and immune system, 85% items were found to have p-value b/w 0.3 -0.7 and 15% had mean p-value 0.2. The mean DI value was 0.41. Out of 40 Physiology questions in Semester exam, 82% items had p-value b/w 0.3 – 0.7 and 18 % items had mean p-value 0.18. The mean DI value was 0.28. Table 1 shows the difficulty and discrimination indices of Physiology items in respiratory system. The p-value of 4 items was 0.17, 0.26, 0.27 and 0.22 respectively. The discrimination index of all 4 questions was >0.2. Table 1 also shows the difficulty and discrimination index of Physiology items in Hemopoietic and Immune system. The p-value of 2 items was 0.22 and 0.18 respectively. The discrimination index of 2 items was >0.2.

Table 1: Item analysis of MBBS 1^st Year Semester II CAT I respiratory system and CAT II for hemopoietic and immune system.

Table 2: Item analysis of MBBS 1^st Year Semester II Respiratory and hemopoietic and immune system.

Table 2 shows the difficulty and discrimination indices of Physiology items in semester II. The p-value of 7 items was 0.2, 0.19, 0.15, 0.21, 0.18, 0.17 and 0.15. The mean p-value was 0.18. The discrimination index of 7 items was >0.24. The mean DI was 0.28.

In this study, the item analysis was done on Physiology MCQs included in CATs of courses taught in semester II MBBS as well as the semester II exam. The item analysis was matched with physiology teaching. The same faculty members were asked to develop one best Physiology MCQs on the topics they taught. In respiratory system CAT, 4 out of 15 MCQs had mean p-value of <0.23. In Hemopoietic system, 2 out of 14 MCQs had mean p-value of 0.2. The topics of these 6 MCQs were reviewed in relation to teaching. The quality of continuous assessment tests plays a vital role in giving feedback to teachers on their learning activities¹³. With item analysis, teachers get an idea about how well a particular topic has been comprehended by the students and which topics need more time to go through¹⁵. Based on the finding of the present study, teachers customized their teaching either for content or learning strategy¹⁵. Moreover, item analysis is beneficial for increasing teachers’ skills in MCQ development and indicating particular areas of the course content, which require better emphasis or clarity¹⁴. Discussing the outcome of item analysis with faculty helps in improving learning outcome¹⁵.

Optimizing student learning and academic performance is a constant challenge for medical schools¹⁶. As assessment is an important component of student learning, assessment tools should be valid, reliable and objective and indicate various achievement levels¹². Multiple-choice questions (MCQs) are a tool that is commonly used to evaluate learning performance of undergraduate students¹⁵. In the discipline of Physiology, the concepts, terms, and structures are not readily visualized. The mechanisms need to be taught by employing distinct strategies to enhance comprehension¹⁷. Developing the skill to think critically is a fundamental aspect of undergraduate physiology education¹⁸. It is governed by appropriate use of learning and assessment strategies.

Out of 80 MCQs, there were 40 Physiology MCQs in semester II exam. Developing MCQs to assess students’ knowledge thoroughly at the end of a semester is a difficult and tedioustask¹³. At this point, item analysis provided important diagnostic information on what undergraduate students have learned and what they have not learned¹⁴. This correlation of assessment with teaching learning was a reflection of the students’ performance regarding the learning objectives taught over the specific period of time¹³. Item analysis and storing the MCQs with their indices allows an examiner to select MCQs of appropriate difficulty level according to the need of assessment¹⁶.

The item analysis of the majority of one best type MCQs of CATs and semester exam in Physiology had good difficulty and discrimination indices and matched with teaching and learning in Physiology.

The authors would like to acknowledge the Jinnah Medical and Dental College for facilitating the study.

The authors declare no conflict of interest.

The study was approved from the institutional review board of the Jinnah Medical and Dental College.

Informed written consents were taken from all the participants.

All authors contributed equally in this manuscript.

1. Saint DA, Horton D, Yool A, Elliott A. A progressive assessment strategy improves student learning and perceived course quality in undergraduate physiology. Adv Physiol Educ. 2015; 39(3):218-222
2. Ferris H, Flynn DO. Assessment in medical education; what are we trying to achieve? Int J Higher Educ. 2015; 4(2):139-144.
3. Tariq S, Tariq S, Maqsood S, Jawed S, Baig M. Evaluation of cognitive levels and item writing flaws in medical pharmacology internal assessment examinations. Pak J Med Sci. 2017; 33(4):866-870.
4. Neilson DG, Gotzsche O, Sonne O, Eika B. The relationship between immediate relevant basic science knowledge and clinical knowledge: physiology knowledge and transthoracic echocardiography image interpretation. Adv Health Sci Educ. 2012;17 (4):501-513
5. Ghosh S, Dawka V. Combination of didactic lecture with problem-based learning sessions in physiology teaching in a developing medical college in Nepal. Adv Physiol Educ. 2000; 24(1):8-12
6. Bhattacharya J, Chakraborty S, Thakurta TG. Learning gains through multiple choice questions in physiology. Int J Contemp Med Res. 2017; 4(1):254-260.
7. Islam ZU, Usmani A. Psychometric analysis of anatomy MCQs in modular examination. Pak J Med Sci. 2017;33 (5):1138-1143.
8. Hingorjo M, Jaleel F. Analysis of one-best MCQs: the difficulty index, discrimination index and distractor efficiency. J Pak Med Assoc. 2012; 62 (2):142-147.
9. Shahid R, Farooq Q, Iqbal R. Item analysis of multiple choice questions of ophthalmology at Rawalpindi Medical University, Rawalpindi, Pakistan. Rawal Med J. 2019; 44(1):192-195.
10. Burud I, Nagandla K, Agarwal P. Impact of distractors in item analysis of multiple choice questions. Int J Res Med Sci. 2019;7(4):1136-1139.
11. Shete AN, Kausar A, Lakhkar K, Khan ST. Item analysis: An evaluation of multiple choice questions in Physiology examination. J Contemp Med Educ. 2015; 3(3):106-109.
12. Kheyami D, Jaradat A, Al-Shibani T, Ali FA. Item analysis of multiple choice questions at the department of paediatrics, Arabian Gulf University, Manama, Bahrain. Sultan Qaboos Univ Med J. 2018; 18(1):e68-e74.
13. Quaigrain K, Arhin AK. Using reliability and item analysis to evaluate a teacher-developed test in educational measurement and evaluation. Cogent Educ. 2017;4:1-11.
14. Boopathiraj C, Chellamani K. Analysis of test items on difficulty level and discrimination index in the test for research in education. Int J Soc Sci Interdiscip Res. 2013; 2(2):189-193.
15. Kolte V. item analysis of multiple choice questions in physiology examination. Int J Appl Basic Med Res. 2015;4(4):320-326.
16. Kickert R, Stegers‐Jager KM, Meeuwisse M, Prinzie P, Arends LR. The role of the assessment policy in the relation between learning and performance. Med Educ. 2018;52(3):324-335.
17. Rehman R, Khan AN, Kamran A. Role of small group interactive sessions in two different curriculums based medical colleges. J Pak Med Assoc. 2012; 62(9):920-923.
18. Abraham RR, Upadhya S, Torke S, Ramnarayan K. Clinically oriented physiology teaching: strategy for developing critical-thinking skills in undergraduate medical students. Adv Physiol Educ. 2004; 28 (1-4):102-104.