Comparative Analysis: Global Rating Scale vs. Checklist in Teaching and Assessing Skill Competence

By Somia Saghir¹, Bushra Sultan¹, Farhana¹

DOI: https://doi.org/10.36283/PJMD13-2/015

How to cite: Saghir S, Sultan B, Farhana. Comparative Analysis: Global Rating Scale vs. Checklist in Teaching and Assessing Skill Competence. Pak J Med Dent. 2024;13(2): 102-105. Doi: 10.36283/PJMD13-2/015

This study compared the effectiveness of the Global Rating Scale (GRS) versus the checklist for teaching and evaluating nursing students on two skills; nasogastric tube (NGT) and subcutaneous injection. Using a census sample, 100 students of 2^nd year Bachelor of Science in Nursing (BSN) were enrolled and divided into two groups. The control group was taught and evaluated on the traditional checklist, whereas the experimental group through the GRS. The results showed students in the experimental group obtained a higher mean score (NGT 9.41, subcutaneous injection 9.27) than the control group (NGT 7.2, subcutaneous injection 7.6). Likewise, critical point scores were also notably higher in the experimental group (NGT 9.6, subcutaneous injection 9.8) than in the control group (NGT 1.7, subcutaneous injection 2.4). GRS is recommended for teaching and evaluating nursing students’ psychomotor skills. Educator training is essential for effective GRS utilization, enhancing performance evaluation, ensuring competence, and aligning with study program objectives for enhanced patient safety.

Keywords: COPD, Hospital mortality, Non-invasive Ventilation, Risk factors, Symptom Exacerbation.

In health professional education, teaching learning and assessment tools play a crucial role in gauging students’ proficiency and monitoring their development. Checklists and global rating scales (GRSs), two significant tools among the variety of teaching learning and assessment methods available, have long been the subject of controversy. GRSs focus on overall performance and global impressions, while checklists use a more structured approach and emphasize observable actions. Clinical nursing is of paramount importance in nursing education. Nurses are the only medical personnel who spend more time with patients than any other. For this reason, a lack of competency can harm patient safety and outcomes¹. A systematic review of the checklists versus global rating scales in simulation-based assessment was done. They evaluated the effectiveness of the GRS in improving nursing performance. The shreds of evidence showed a significant improvement in nursing performance after implementing the GRS. Therefore, suggested that GRS can be used across multiple tasks, and may better capture nuanced elements of expertise ².

In addition to this, the clinical competency of students is rated by lecturers with a high degree of agreement. Moreover, GRS may assist participants in obtaining more meaningful information about the patient rather than recognizing conventional disease concerns, particularly when working with standardized patients³. The GRS looks to be an effective tool for identifying service deficiencies during patient engagement. Checklists have little validity and reliability when it comes to assessing skills. Therefore, it has been suggested that the checklist should be replaced with GRS⁴. GRS is applicable in alternative healthcare settings but, unfortunately, it has been noticed that the utilization is high in general education as compared to clinical practice⁴.

The comparison between GRS and Checklist as teaching learning and assessment tools for nursing students is a crucial area of research. While both tools are commonly used to evaluate communication skills, the effectiveness and validity of each method have not been fully explored. Therefore, this study aimed to compare the scores of nursing students while using the global rating scale and checklist.

A global rating scale and checklist were used in the teaching, learning, and assessment phases. All the second-year nursing students (n=100) in a private college enrolled in the Adult Heath Nursing course participated in this project. Participants were divided into two groups (50 each).  Two skills were selected to compare the effectiveness. Informed consent of the students was secured.

In the experimental group: The facilitator demonstrates the nasogastric tube (NGT) skill in front of students using the GRS. Students practiced the skill for 3 hours in small groups while using the same GRS. After one week the teacher signs off each student keeping the same GRS as an assessment tool. The same process was followed for the subcutaneous injection.

In the control group: The facilitator demonstrates the NGT skill in front of students using the skill checklist. Students practiced the skill for 3 hours in small groups while using the same checklist. After one week the teachers sign off each student keeping the same checklist as an assessment tool. The same process was followed for the subcutaneous injection.  Four educators evaluated the experimental group and four evaluated the control group. Moreover, the same tools were used in the Objective Structured Clinical Examination for both groups. Educators compiled the demographic information shortly after signing off students. The control group was exposed to the GRS after the data collection period.

GRS was developed on NGT (insertion feeding and removal), and subcutaneous injection by the first and second authors. Extensive literature was reviewed for developing GRS. The GRS was sent to five experts-three clinical instructors and two educators. Experts were instructed to not only delete but also add relevant points and assign the scores to each point based on their importance. After the first round, all the expert feedback was integrated and highlighted into one modified list. In round two, GRS was sent back to the experts for confirmation and to develop a consensus on items and scoring. Internal consistency and inter-rater reliability were maintained. Face, construct, and content validity were maintained by faculty members were had 5 years of teaching experience. GRS was pilot-tested on 10% of the population.

A checklist of both skills was traditionally used. Two rating scales are used based on the critical and non-critical points on each checklist. For non-critical points, a 0-2 scale is employed, 0= not performed, 1= needs improvement, and 2=satisfactorily completed. Moreover, for critical points, 0 or 5 scale was used, 0= not performed, and 5= properly completed. The total points on NGT and subcutaneous skills are 19 and 23. The maximum point of NGT is ‘45’, on subcutaneous is ’80’ and the minimum is ‘0’. The total mark required to pass this skill is 50%, whether the students perform critical points correctly or not.

In contrast, GRS, using a 0–2 rating scale was used on both skills, 0=not done 1=need improvement, and 2= done correctly. The maximum point a student can achieve on NGT is ‘50’, on subcutaneous skill is ‘30’ marks and the minimum is ‘0’. Safety is regarded as the highest priority in the GRS and is given a weightage that is significantly higher than the overall points.  Data was analyzed by using the SPSS version 21. Frequencies and percentages were used for the demographic data. Moreover, the t-test was used to compare the scores after checking the normality.

The experimental and control groups were each comprised of 100 BSN nursing students. The students ranged in age from 19 to 24. 32% of the students in the experimental group were male, while 68% were female. Similarly, 57% of female students and 43% of male students were in the control group. (Table #1).

Table 1: Demographic Variables.

Findings revealed that the overall mean scores of nasogastric tube skill 9.41+1.26 while using the GRS scale was higher than scores in the same skill while using checklist 7.2+0.94. Similarly, the overall mean scores in the subcutaneous injection skill while using the GRS were higher 9.27+1.21 compared to scores in the same skill while using the checklist 7.6+0.78 (Table 2).  Moreover, there was a statistically significant difference (p= 0.000) in the scores while using a GRS and checklist in both skills.

Table 2: Comparison of scores in NGT skill and subcutaneous skill while using GRS and Checklist.

In consideration of the safety/ critical points, mean scores were higher in the critical/safety point in the nasogastric skill 9.6+1.96 while using the GRS as compared to checklist 1.7+3.1. Likewise, the mean scores in the critical/safety point of subcutaneous injection administration skill were higher at 9.8+1.21 in comparison to the checklist at 2.4+0.78. (Table 2). Additionally, there was a statistically significant difference in scores while using a GRS and checklist (p=0.000) in both skills.

Checklists and GRS are currently widely employed in health professions education. Moreover, it has been used in clinical simulation-based teaching and assessment as extensively discussed in the literature^5,7. The study findings revealed that the experimental group (who were taught via the GRS tool during skill demonstration and finally assessed through GRS) performance was enhanced than the non-experimental group.  This finding is consistent with existing literature. GRS and checklist have been used for the Infant Lumbar Puncture Procedure. The result showed the enhancement of the performance while using GRS⁸. Similarly, the finding indicated that the GRS scores were higher in the experimental group as compared to the checklist⁹.

Additionally, the mean scores in the critical/safety point in both skills were higher in the experimental group. This might be due to the higher weightage of safety/critical points which is more than 50% in comparison to other or non-critical points in GRS allows students to be vigilant and careful in performance. Moreover, it prevents them from falling into a particular skill. In consistency, the use of modern checklists, where each item carries a different weight, has resulted in validated evaluation results that are superior to those obtained by the checklist¹⁰.

Likewise, the reliability between items and between stations in the global rating approach was observed to be higher than that in the checklist method resulting in a more accurate assessment of skills².  In contrast, it is challenging for the faculty to check off the checklist due to the increased number of critical points and the weightage assigned to each critical item. Moreover, the scoring of each critical point is not very much higher than the other non-critical points. However, students get an advantage of passing the skill without remaining careful about the safety aspect of the patients which has a negative implication on the competence level. Undoubtedly, this makes it easy to grade students and decide whether they pass or fail based on the highest weightage of the critical points.

GRS was found to be an effective tool in enhancing the skill competence of nursing students. It should be used since it allows teachers to efficiently use their teaching, learning, and assessment process to grade passes or fails owing to the higher weighting of the safety points. It is effective in enhancing the skill competence of nursing students. GRS should be implemented on multiple skills and in different academic years to establish generalizability. Faculty members should be trained enough in the development and utilization of the GRS skills.

We would like to express our sincere appreciation to all the national and international reviewers for dedicating their valuable time to refining the manuscript and bringing it to its current stage.

The authors declared no conflict of interest.

SS: Data collection, GRS tool construction, Study Design, Manuscript, and Literature Review. BS: Data Collection, GRS tool construction, Study Design, Data Analysis, interpretations of the results, and Critical Review of the overall project. F: Data Collection, GRS tool construction, Data entry.

1. Aronowitz T, Aronowitz S, Mardin-Small J, Kim BR. Using Objective Structured Clinical Examination (OSCE) as Education in Advanced Practice Registered Nursing Education. J Prof Nurs. 2017;33(2):119–125. https://doi.org/10.1016/j.profnurs.2016.06.003
2. Ilgen JS, Ma IWY, Hatala R, Cook DA. A systematic review of validity evidence for checklists versus global rating scales in simulation-based assessment. Med Educ. 2015;49(2):161–173. https://doi.org/10.1111/medu.12621
3. Bremer A, Andersson Hagiwara M, Tavares W, Paakkonen H, Nyström P, Andersson H. Translation and further validation of a global rating scale for the assessment of clinical competence in prehospital emergency care. Nurse Educ Pract. 2020;47. https://doi.org/10.1016/j.nepr.2020.102841
4. Sim JH, Abdul Aziz YF, Vijayanantha A, Mansor A, Vadivelu J, Hassan H. A Closer Look at Checklist Scoring and Global Rating for Four OSCE Stations: Do the Scores Correlate Well? Educ Med J. 2015;7(2):39–44. DOI: 10.5959/eimj.v7i2.341
5. Henrico K. The Use of Global Rating Scales and Checklists in Clinical Simulation-Based Assessments: A Protocol for a Scoping Review Scoping review protocol registration: 2021;1–13. https://doi.org/10.1136/bmjopen-2022-065981
6. Greatorex J, Johnson M, Coleman V. A review of instruments for assessing complex vocational competence. Res Matters; A Cambridge Assess Publ. 2017;(23):35–42. https://doi.org/10.17863/CAM.100353
7. Zoller A, Hölle T, Wepler M, Radermacher P, Nussbaum BL. Development of a novel global rating scale for objective structured assessment of technical skills in an emergency medical simulation training. BMC Med Educ. 2021;21(1):1–8. https://doi.org/10.1186/s12909-021-02580-4
8. Gerard JM, Kessler DO, Braun C, Mehta R, Scalzo AJ, Auerbach M. Validation of global rating scale and checklist instruments for the infant lumbar puncture procedure. Simul Healthc. 2013;8(3):148–154. DOI: 10.1097/SIH.0b013e3182802d34
9. Wong DM, Watson MJ, Kluger R, Chuan A, Herrick MD, Ng I, et al. Evaluation of a task-specific checklist and global rating scale for ultrasound-guided regional anesthesia. Reg AnesthPainMed.2014;39(5):399–408. https://doi.org/10.1097/AAP.0000000000000126
10. Muthusami A, Mohsina S, Sureshkumar S, Anandhi A, Elamurugan TP, Srinivasan K, et al. Efficacy and Feasibility of Objective Structured Clinical Examination in the Internal Assessment for Surgery Postgraduates. J Surg Educ. 2017;74(3):398–405. http://dx.doi.org/10.1016/j.jsurg.2016.11.004

This is an open-access article distributed under the terms of the CreativeCommons Attribution License (CC BY) 4.0 https://creativecommons.org/licenses/by/4.0/