Loading…
Bias Analysis Gone Bad
Abstract Quantitative bias analysis comprises the tools used to estimate the direction, magnitude, and uncertainty from systematic errors affecting epidemiologic research. Despite the availability of methods and tools, and guidance for good practices, few reports of epidemiologic research incorporat...
Saved in:
Published in: | American journal of epidemiology 2021-08, Vol.190 (8), p.1604-1612 |
---|---|
Main Authors: | , , , , |
Format: | Article |
Language: | English |
Subjects: | |
Citations: | Items that this one cites Items that cite this one |
Online Access: | Get full text |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
cited_by | cdi_FETCH-LOGICAL-c440t-bbd81d0dd83623b422a81737346a68d14e85460fe69d44753f6e6e0c3cdef2813 |
---|---|
cites | cdi_FETCH-LOGICAL-c440t-bbd81d0dd83623b422a81737346a68d14e85460fe69d44753f6e6e0c3cdef2813 |
container_end_page | 1612 |
container_issue | 8 |
container_start_page | 1604 |
container_title | American journal of epidemiology |
container_volume | 190 |
creator | Lash, Timothy L Ahern, Thomas P Collin, Lindsay J Fox, Matthew P MacLehose, Richard F |
description | Abstract
Quantitative bias analysis comprises the tools used to estimate the direction, magnitude, and uncertainty from systematic errors affecting epidemiologic research. Despite the availability of methods and tools, and guidance for good practices, few reports of epidemiologic research incorporate quantitative estimates of bias impacts. The lack of familiarity with bias analysis allows for the possibility of misuse, which is likely most often unintentional but could occasionally include intentional efforts to mislead. We identified 3 examples of suboptimal bias analysis, one for each common bias. For each, we describe the original research and its bias analysis, compare the bias analysis with good practices, and describe how the bias analysis and research findings might have been improved. We assert no motive to the suboptimal bias analysis by the original authors. Common shortcomings in the examples were lack of a clear bias model, computed example, and computing code; poor selection of the values assigned to the bias model’s parameters; and little effort to understand the range of uncertainty associated with the bias. Until bias analysis becomes more common, community expectations for the presentation, explanation, and interpretation of bias analyses will remain unstable. Attention to good practices should improve quality, avoid errors, and discourage manipulation. |
doi_str_mv | 10.1093/aje/kwab072 |
format | article |
fullrecord | <record><control><sourceid>proquest_pubme</sourceid><recordid>TN_cdi_pubmedcentral_primary_oai_pubmedcentral_nih_gov_8484933</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><oup_id>10.1093/aje/kwab072</oup_id><sourcerecordid>2506514814</sourcerecordid><originalsourceid>FETCH-LOGICAL-c440t-bbd81d0dd83623b422a81737346a68d14e85460fe69d44753f6e6e0c3cdef2813</originalsourceid><addsrcrecordid>eNp90E1LAzEQBuAgiq3Vk-BRCoIIsnbynb0IbdEqFLzoOWQ3Wd263dRNV-m_N9Ja1IOnOczDy8yL0DGGKwwpHZiZG7x-mAwk2UFdzKRIBOFiF3UBgCQpEaSDDkKYAWCccthHHUqlVIrxLjoZlSb0h7WpVqEM_YmvXX9k7CHaK0wV3NFm9tDT7c3j-C6ZPkzux8NpkjMGyyTLrMIWrFVUEJoxQozCkkrKhBHKYuYUZwIKJ1LLmOS0EE44yGluXUEUpj10vc5dtNnc2dzVy8ZUetGUc9OstDel_r2pyxf97N-1YoqllMaAi01A499aF5Z6XobcVZWpnW-DJhwEx0xhFunZHzrzbRM_j0oxwiWOLqrLtcobH0Ljiu0xGPRX3zr2rTd9R3368_6t_S44gvM18O3i36RPZmGG1g</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2842571148</pqid></control><display><type>article</type><title>Bias Analysis Gone Bad</title><source>Oxford Journals Online</source><creator>Lash, Timothy L ; Ahern, Thomas P ; Collin, Lindsay J ; Fox, Matthew P ; MacLehose, Richard F</creator><creatorcontrib>Lash, Timothy L ; Ahern, Thomas P ; Collin, Lindsay J ; Fox, Matthew P ; MacLehose, Richard F</creatorcontrib><description>Abstract
Quantitative bias analysis comprises the tools used to estimate the direction, magnitude, and uncertainty from systematic errors affecting epidemiologic research. Despite the availability of methods and tools, and guidance for good practices, few reports of epidemiologic research incorporate quantitative estimates of bias impacts. The lack of familiarity with bias analysis allows for the possibility of misuse, which is likely most often unintentional but could occasionally include intentional efforts to mislead. We identified 3 examples of suboptimal bias analysis, one for each common bias. For each, we describe the original research and its bias analysis, compare the bias analysis with good practices, and describe how the bias analysis and research findings might have been improved. We assert no motive to the suboptimal bias analysis by the original authors. Common shortcomings in the examples were lack of a clear bias model, computed example, and computing code; poor selection of the values assigned to the bias model’s parameters; and little effort to understand the range of uncertainty associated with the bias. Until bias analysis becomes more common, community expectations for the presentation, explanation, and interpretation of bias analyses will remain unstable. Attention to good practices should improve quality, avoid errors, and discourage manipulation.</description><identifier>ISSN: 0002-9262</identifier><identifier>EISSN: 1476-6256</identifier><identifier>DOI: 10.1093/aje/kwab072</identifier><identifier>PMID: 33778845</identifier><language>eng</language><publisher>United States: Oxford University Press</publisher><subject>Antidepressive Agents - adverse effects ; Bias ; Breast Neoplasms - chemically induced ; Contraceptive Agents, Hormonal - adverse effects ; Data Interpretation, Statistical ; Editor's Choice ; Epidemiologic Studies ; Epidemiology ; Familiarity ; Humans ; Marijuana Abuse - complications ; Mental Disorders - etiology ; Models, Statistical ; Practice of Epidemiology ; Reproducibility of Results ; Research Design - standards ; Systematic errors ; Uncertainty</subject><ispartof>American journal of epidemiology, 2021-08, Vol.190 (8), p.1604-1612</ispartof><rights>The Author(s) 2021. Published by Oxford University Press on behalf of the Johns Hopkins Bloomberg School of Public Health. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com. 2021</rights><rights>The Author(s) 2021. Published by Oxford University Press on behalf of the Johns Hopkins Bloomberg School of Public Health. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c440t-bbd81d0dd83623b422a81737346a68d14e85460fe69d44753f6e6e0c3cdef2813</citedby><cites>FETCH-LOGICAL-c440t-bbd81d0dd83623b422a81737346a68d14e85460fe69d44753f6e6e0c3cdef2813</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>230,314,780,784,885,27924,27925</link.rule.ids><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/33778845$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><creatorcontrib>Lash, Timothy L</creatorcontrib><creatorcontrib>Ahern, Thomas P</creatorcontrib><creatorcontrib>Collin, Lindsay J</creatorcontrib><creatorcontrib>Fox, Matthew P</creatorcontrib><creatorcontrib>MacLehose, Richard F</creatorcontrib><title>Bias Analysis Gone Bad</title><title>American journal of epidemiology</title><addtitle>Am J Epidemiol</addtitle><description>Abstract
Quantitative bias analysis comprises the tools used to estimate the direction, magnitude, and uncertainty from systematic errors affecting epidemiologic research. Despite the availability of methods and tools, and guidance for good practices, few reports of epidemiologic research incorporate quantitative estimates of bias impacts. The lack of familiarity with bias analysis allows for the possibility of misuse, which is likely most often unintentional but could occasionally include intentional efforts to mislead. We identified 3 examples of suboptimal bias analysis, one for each common bias. For each, we describe the original research and its bias analysis, compare the bias analysis with good practices, and describe how the bias analysis and research findings might have been improved. We assert no motive to the suboptimal bias analysis by the original authors. Common shortcomings in the examples were lack of a clear bias model, computed example, and computing code; poor selection of the values assigned to the bias model’s parameters; and little effort to understand the range of uncertainty associated with the bias. Until bias analysis becomes more common, community expectations for the presentation, explanation, and interpretation of bias analyses will remain unstable. Attention to good practices should improve quality, avoid errors, and discourage manipulation.</description><subject>Antidepressive Agents - adverse effects</subject><subject>Bias</subject><subject>Breast Neoplasms - chemically induced</subject><subject>Contraceptive Agents, Hormonal - adverse effects</subject><subject>Data Interpretation, Statistical</subject><subject>Editor's Choice</subject><subject>Epidemiologic Studies</subject><subject>Epidemiology</subject><subject>Familiarity</subject><subject>Humans</subject><subject>Marijuana Abuse - complications</subject><subject>Mental Disorders - etiology</subject><subject>Models, Statistical</subject><subject>Practice of Epidemiology</subject><subject>Reproducibility of Results</subject><subject>Research Design - standards</subject><subject>Systematic errors</subject><subject>Uncertainty</subject><issn>0002-9262</issn><issn>1476-6256</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2021</creationdate><recordtype>article</recordtype><recordid>eNp90E1LAzEQBuAgiq3Vk-BRCoIIsnbynb0IbdEqFLzoOWQ3Wd263dRNV-m_N9Ja1IOnOczDy8yL0DGGKwwpHZiZG7x-mAwk2UFdzKRIBOFiF3UBgCQpEaSDDkKYAWCccthHHUqlVIrxLjoZlSb0h7WpVqEM_YmvXX9k7CHaK0wV3NFm9tDT7c3j-C6ZPkzux8NpkjMGyyTLrMIWrFVUEJoxQozCkkrKhBHKYuYUZwIKJ1LLmOS0EE44yGluXUEUpj10vc5dtNnc2dzVy8ZUetGUc9OstDel_r2pyxf97N-1YoqllMaAi01A499aF5Z6XobcVZWpnW-DJhwEx0xhFunZHzrzbRM_j0oxwiWOLqrLtcobH0Ljiu0xGPRX3zr2rTd9R3368_6t_S44gvM18O3i36RPZmGG1g</recordid><startdate>20210801</startdate><enddate>20210801</enddate><creator>Lash, Timothy L</creator><creator>Ahern, Thomas P</creator><creator>Collin, Lindsay J</creator><creator>Fox, Matthew P</creator><creator>MacLehose, Richard F</creator><general>Oxford University Press</general><general>Oxford Publishing Limited (England)</general><scope>CGR</scope><scope>CUY</scope><scope>CVF</scope><scope>ECM</scope><scope>EIF</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7QP</scope><scope>7T2</scope><scope>7TK</scope><scope>7U7</scope><scope>7U9</scope><scope>C1K</scope><scope>H94</scope><scope>K9.</scope><scope>NAPCQ</scope><scope>7X8</scope><scope>5PM</scope></search><sort><creationdate>20210801</creationdate><title>Bias Analysis Gone Bad</title><author>Lash, Timothy L ; Ahern, Thomas P ; Collin, Lindsay J ; Fox, Matthew P ; MacLehose, Richard F</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c440t-bbd81d0dd83623b422a81737346a68d14e85460fe69d44753f6e6e0c3cdef2813</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2021</creationdate><topic>Antidepressive Agents - adverse effects</topic><topic>Bias</topic><topic>Breast Neoplasms - chemically induced</topic><topic>Contraceptive Agents, Hormonal - adverse effects</topic><topic>Data Interpretation, Statistical</topic><topic>Editor's Choice</topic><topic>Epidemiologic Studies</topic><topic>Epidemiology</topic><topic>Familiarity</topic><topic>Humans</topic><topic>Marijuana Abuse - complications</topic><topic>Mental Disorders - etiology</topic><topic>Models, Statistical</topic><topic>Practice of Epidemiology</topic><topic>Reproducibility of Results</topic><topic>Research Design - standards</topic><topic>Systematic errors</topic><topic>Uncertainty</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Lash, Timothy L</creatorcontrib><creatorcontrib>Ahern, Thomas P</creatorcontrib><creatorcontrib>Collin, Lindsay J</creatorcontrib><creatorcontrib>Fox, Matthew P</creatorcontrib><creatorcontrib>MacLehose, Richard F</creatorcontrib><collection>Medline</collection><collection>MEDLINE</collection><collection>MEDLINE (Ovid)</collection><collection>MEDLINE</collection><collection>MEDLINE</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>Calcium & Calcified Tissue Abstracts</collection><collection>Health and Safety Science Abstracts (Full archive)</collection><collection>Neurosciences Abstracts</collection><collection>Toxicology Abstracts</collection><collection>Virology and AIDS Abstracts</collection><collection>Environmental Sciences and Pollution Management</collection><collection>AIDS and Cancer Research Abstracts</collection><collection>ProQuest Health & Medical Complete (Alumni)</collection><collection>Nursing & Allied Health Premium</collection><collection>MEDLINE - Academic</collection><collection>PubMed Central (Full Participant titles)</collection><jtitle>American journal of epidemiology</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Lash, Timothy L</au><au>Ahern, Thomas P</au><au>Collin, Lindsay J</au><au>Fox, Matthew P</au><au>MacLehose, Richard F</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Bias Analysis Gone Bad</atitle><jtitle>American journal of epidemiology</jtitle><addtitle>Am J Epidemiol</addtitle><date>2021-08-01</date><risdate>2021</risdate><volume>190</volume><issue>8</issue><spage>1604</spage><epage>1612</epage><pages>1604-1612</pages><issn>0002-9262</issn><eissn>1476-6256</eissn><abstract>Abstract
Quantitative bias analysis comprises the tools used to estimate the direction, magnitude, and uncertainty from systematic errors affecting epidemiologic research. Despite the availability of methods and tools, and guidance for good practices, few reports of epidemiologic research incorporate quantitative estimates of bias impacts. The lack of familiarity with bias analysis allows for the possibility of misuse, which is likely most often unintentional but could occasionally include intentional efforts to mislead. We identified 3 examples of suboptimal bias analysis, one for each common bias. For each, we describe the original research and its bias analysis, compare the bias analysis with good practices, and describe how the bias analysis and research findings might have been improved. We assert no motive to the suboptimal bias analysis by the original authors. Common shortcomings in the examples were lack of a clear bias model, computed example, and computing code; poor selection of the values assigned to the bias model’s parameters; and little effort to understand the range of uncertainty associated with the bias. Until bias analysis becomes more common, community expectations for the presentation, explanation, and interpretation of bias analyses will remain unstable. Attention to good practices should improve quality, avoid errors, and discourage manipulation.</abstract><cop>United States</cop><pub>Oxford University Press</pub><pmid>33778845</pmid><doi>10.1093/aje/kwab072</doi><tpages>9</tpages><oa>free_for_read</oa></addata></record> |
fulltext | fulltext |
identifier | ISSN: 0002-9262 |
ispartof | American journal of epidemiology, 2021-08, Vol.190 (8), p.1604-1612 |
issn | 0002-9262 1476-6256 |
language | eng |
recordid | cdi_pubmedcentral_primary_oai_pubmedcentral_nih_gov_8484933 |
source | Oxford Journals Online |
subjects | Antidepressive Agents - adverse effects Bias Breast Neoplasms - chemically induced Contraceptive Agents, Hormonal - adverse effects Data Interpretation, Statistical Editor's Choice Epidemiologic Studies Epidemiology Familiarity Humans Marijuana Abuse - complications Mental Disorders - etiology Models, Statistical Practice of Epidemiology Reproducibility of Results Research Design - standards Systematic errors Uncertainty |
title | Bias Analysis Gone Bad |
url | http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-23T03%3A01%3A07IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_pubme&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Bias%20Analysis%20Gone%20Bad&rft.jtitle=American%20journal%20of%20epidemiology&rft.au=Lash,%20Timothy%20L&rft.date=2021-08-01&rft.volume=190&rft.issue=8&rft.spage=1604&rft.epage=1612&rft.pages=1604-1612&rft.issn=0002-9262&rft.eissn=1476-6256&rft_id=info:doi/10.1093/aje/kwab072&rft_dat=%3Cproquest_pubme%3E2506514814%3C/proquest_pubme%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-c440t-bbd81d0dd83623b422a81737346a68d14e85460fe69d44753f6e6e0c3cdef2813%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_pqid=2842571148&rft_id=info:pmid/33778845&rft_oup_id=10.1093/aje/kwab072&rfr_iscdi=true |