Intra- and inter-rater agreement in the assessment of occupational exposure to metals

• Published in: International journal of epidemiology 1998-04, Vol.27 (2), p.269-273
• Main Authors: Rybicki, Benjamin A, Peterson, Edward L, Johnson, Christine C, Kortsha, Gene X, Cleary, William M, Gorell, Jay M
• Format: Article
• Language: English
• Subjects: Aged; bias; Biological and medical sciences; Brain Diseases - epidemiology; Brain Diseases - etiology; Case-Control Studies; Classification; copper; Copper - adverse effects; Epidemiology; Female; General aspects; Humans; iron; Iron - adverse effects; kappa; lead; Lead - adverse effects; Likelihood Functions; Male; Medical sciences; Metals - adverse effects; Methodology; Michigan - epidemiology; misclassification; Observer Variation; Occupational Diseases - epidemiology; Occupational Diseases - etiology; Occupational Exposure - statistics & numerical data; Odds Ratio; Public health. Hygiene; Public health. Hygiene-occupational medicine; reliability; Reproducibility of Results; Risk Factors
• ISSN: 0300-5771; 1464-3685; 1464-3685
• DOI: 10.1093/ije/27.2.269

Background Among the methods of retrospective occupational exposure assessment, expert review is considered the most accurate. Although expert review provides a more objective measure of exposure, depending on the exposure of interest it may still result in a significant degree of misclassification. Methods To evaluate the reliability in occupational metal exposure assessment by expert review, we analysed job history data from a case-control study of a neurological disease. First, one industrial hygienist (IH) did an initial exposure assessment of the metals copper, iron and lead, blinded to case-control status, for 608 subjects who had 3033 total jobs. We then compared exposure assessments from the original review with a second blinded review of 60 job histories (306 jobs) by the same IH (intra-rater) and of 64 job histories (361 jobs) by a different IH (inter-rater). Results The per cent agreements for the intra-IH comparisons were 89.6 for copper, 87.9 for iron and 94.6 for lead, whereas the inter-N per cent agreements were 86.4 for copper, 81.1 for iron and 76.2 for lead. Based on the assumption that reliability is related to validity, we calculated an estimate of misclassification of metal exposure by one LIT. Our exposure misclassification estimates show a sizable attenuation of the odds ratio, with the expected bias similar for copper and iron when using either intra- or inter-reliability results to estimate misclassification. Conclusions Our results suggest that variation in the expert assessment of metal exposure is due mainly to the difficulties involved in transforming an occupational history into an estimate of exposure.