Interrater reliability of endoscopic parameters following sinus surgery

• Published in: The Laryngoscope 2012-01, Vol.122 (1), p.230-236
• Main Authors: Smith, Timothy L., Hwang, Peter H., Murr, Andrew H., Lavigne, Francois, Koreck, Andrea
• Format: Article
• Language: English
• Subjects: Biological and medical sciences; Chronic Disease; Chronic rhinosinusitis; endoscopic grading; Endoscopy - statistics & numerical data; Humans; interrater reliability; Level of Evidence: 2c; Medical sciences; Non tumoral diseases; Observer Variation; Otorhinolaryngology (head neck, general aspects and miscellaneous); Otorhinolaryngology. Stomatology; Prospective Studies; Reproducibility of Results; Retrospective Studies; Rhinitis - complications; Rhinitis - surgery; sinus surgery; Sinusitis - complications; Sinusitis - surgery; Upper respiratory tract, upper alimentary tract, paranasal sinuses, salivary glands: diseases, semeiology; visual analog scale
• ISSN: 0023-852X; 1531-4995
• DOI: 10.1002/lary.22440

Objectives/Hypothesis: To determine the interrater reliability of a set of postoperative endoscopic scoring parameters in patients with chronic rhinosinusitis who have undergone endoscopic sinus surgery (ESS). Study Design: Prospective cohort with retrospective review. Methods: One hundred twenty video‐endoscopic evaluations in 20 subjects recorded at 14, 30, and 45 days after ESS were scored in real time by the clinical investigators who performed the endoscopies and recorded the videos and retrospectively by an independent panel of four sinus surgeons who were blinded to all information. The scoring parameters included categoric grading for adhesion formation and middle turbinate position and continuous grading (visual analog scale) for degree of inflammation and crusting. Interrater reliability of the panel members was assessed using the Fleiss kappa test, bias index and prevalence index for categoric data, and the Shrout‐Fleiss test for continuous data. The level of agreement between the panel and the real‐time clinical investigator was also assessed. Results: For categoric variables, strong agreement between raters on the panel was found for both middle turbinate position (kappa = 0.499, prevalence index = 0.925) and adhesions (kappa = 0.364, prevalence index = 0.829). For continuous data, good agreement between raters was found for both inflammation (reliability coefficient = 0.554) and crusting (reliability coefficient = 0.620). Real‐time investigator scoring and panel scoring showed strong agreement. Conclusions: These results suggest that the endoscopic scoring parameters assessed (middle turbinate position, adhesions, inflammation, and crusting) have acceptable interexaminer reproducibility and are suitable for evaluating ESS outcomes in the postsurgical period.