Investigating disagreement in the scientific literature

Disagreement is essential to scientific progress but the extent of disagreement in science, its evolution over time, and the fields in which it happens remain poorly understood. Here we report the development of an approach based on cue phrases that can identify instances of disagreement in scientif...

Full description

Saved in:
Bibliographic Details
Published in:eLife 2021-12, Vol.10
Main Authors: Lamers, Wout S, Boyack, Kevin, Larivière, Vincent, Sugimoto, Cassidy R, van Eck, Nees Jan, Waltman, Ludo, Murray, Dakota
Format: Article
Language:English
Subjects:
Bibliographic coupling
Bibliometrics
citation analysis
Citations
disagreement
Gravitational waves
Interprofessional Relations
Machine learning
Meta-Research
metascience
Natural Language Processing
Natural Science Disciplines
Science
Social Sciences
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by cdi_FETCH-LOGICAL-c4567-52f252f25d778cedf182f441d6f1728d2b1c44b02ac46687f4353154ba9865a23
cites cdi_FETCH-LOGICAL-c4567-52f252f25d778cedf182f441d6f1728d2b1c44b02ac46687f4353154ba9865a23
container_end_page
container_issue
container_start_page
container_title eLife
container_volume 10
creator Lamers, Wout S
Boyack, Kevin
Larivière, Vincent
Sugimoto, Cassidy R
van Eck, Nees Jan
Waltman, Ludo
Murray, Dakota
description Disagreement is essential to scientific progress but the extent of disagreement in science, its evolution over time, and the fields in which it happens remain poorly understood. Here we report the development of an approach based on cue phrases that can identify instances of disagreement in scientific articles. These instances are sentences in an article that cite other articles. Applying this approach to a collection of more than four million English-language articles published between 2000 and 2015 period, we determine the level of disagreement in five broad fields within the scientific literature (biomedical and health sciences; life and earth sciences; mathematics and computer science; physical sciences and engineering; and social sciences and humanities) and 817 meso-level fields. Overall, the level of disagreement is highest in the social sciences and humanities, and lowest in mathematics and computer science. However, there is considerable heterogeneity across the meso-level fields, revealing the importance of local disciplinary cultures and the epistemic characteristics of disagreement. Analysis at the level of individual articles reveals notable episodes of disagreement in science, and illustrates how methodological artifacts can confound analyses of scientific texts.
doi_str_mv 10.7554/eLife.72737
format article
fullrecord <record><control><sourceid>proquest_doaj_</sourceid><recordid>TN_cdi_doaj_primary_oai_doaj_org_article_ad971edf80b541bc9b345a94def68930</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><doaj_id>oai_doaj_org_article_ad971edf80b541bc9b345a94def68930</doaj_id><sourcerecordid>2614225334</sourcerecordid><originalsourceid>FETCH-LOGICAL-c4567-52f252f25d778cedf182f441d6f1728d2b1c44b02ac46687f4353154ba9865a23</originalsourceid><addsrcrecordid>eNpdkc1rHCEYh6W0NCHJqfcy0EuhbOLH66iXQglNsrDQSwu9iaOvE5fZmVRnAvnva3eTkFQQvx4eX_0R8oHRcyUlXOAmRTxXXAn1hhxzKumKavj99sX8iJyVsqW1KdCamffkSICRTGp9TNR6vMcyp97NaeybkIrrM-IOx7lJYzPfYlN8qqsUk2-GNGN285LxlLyLbih49jiekF9X339e3qw2P67Xl982Kw-yVSvJI9_3oJT2GCLTPAKw0EamuA68Yx6go9x5aFutIggpmITOGd1Kx8UJWR-8YXJbe5fTzuUHO7lk9xtT7q3Lc_IDWheMYvUKTTsJrPOmEyCdgYCx1UbQ6vp6cN0t3Q6Dr6_KbnglfX0yplvbT_dWK2qkaqvg86MgT3-W-m12l4rHYXAjTkuxvGXAuRQCKvrpP3Q7LXmsX1UpDoxJDqZSXw6Uz1MpGeNzMYzaf_nafb52n2-lP76s_5l9SlP8Bc2YoC8</addsrcrecordid><sourcetype>Open Website</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2624115249</pqid></control><display><type>article</type><title>Investigating disagreement in the scientific literature</title><source>Open Access: PubMed Central</source><source>Publicly Available Content (ProQuest)</source><creator>Lamers, Wout S ; Boyack, Kevin ; Larivière, Vincent ; Sugimoto, Cassidy R ; van Eck, Nees Jan ; Waltman, Ludo ; Murray, Dakota</creator><creatorcontrib>Lamers, Wout S ; Boyack, Kevin ; Larivière, Vincent ; Sugimoto, Cassidy R ; van Eck, Nees Jan ; Waltman, Ludo ; Murray, Dakota</creatorcontrib><description>Disagreement is essential to scientific progress but the extent of disagreement in science, its evolution over time, and the fields in which it happens remain poorly understood. Here we report the development of an approach based on cue phrases that can identify instances of disagreement in scientific articles. These instances are sentences in an article that cite other articles. Applying this approach to a collection of more than four million English-language articles published between 2000 and 2015 period, we determine the level of disagreement in five broad fields within the scientific literature (biomedical and health sciences; life and earth sciences; mathematics and computer science; physical sciences and engineering; and social sciences and humanities) and 817 meso-level fields. Overall, the level of disagreement is highest in the social sciences and humanities, and lowest in mathematics and computer science. However, there is considerable heterogeneity across the meso-level fields, revealing the importance of local disciplinary cultures and the epistemic characteristics of disagreement. Analysis at the level of individual articles reveals notable episodes of disagreement in science, and illustrates how methodological artifacts can confound analyses of scientific texts.</description><identifier>ISSN: 2050-084X</identifier><identifier>EISSN: 2050-084X</identifier><identifier>DOI: 10.7554/eLife.72737</identifier><identifier>PMID: 34951588</identifier><language>eng</language><publisher>England: eLife Sciences Publications Ltd</publisher><subject>Bibliographic coupling ; Bibliometrics ; citation analysis ; Citations ; disagreement ; Gravitational waves ; Interprofessional Relations ; Machine learning ; Meta-Research ; metascience ; Natural Language Processing ; Natural Science Disciplines ; Science ; Social Sciences</subject><ispartof>eLife, 2021-12, Vol.10</ispartof><rights>2021, Lamers et al.</rights><rights>2021, Lamers et al. This work is published under https://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><rights>2021, Lamers et al 2021 Lamers et al</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c4567-52f252f25d778cedf182f441d6f1728d2b1c44b02ac46687f4353154ba9865a23</citedby><cites>FETCH-LOGICAL-c4567-52f252f25d778cedf182f441d6f1728d2b1c44b02ac46687f4353154ba9865a23</cites><orcidid>0000-0001-7814-8951 ; 0000-0001-7176-9579 ; 0000-0001-8448-4521 ; 0000-0001-8249-1752 ; 0000-0002-7119-0169 ; 0000-0002-2733-0689</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://www.proquest.com/docview/2624115249/fulltextPDF?pq-origsite=primo$$EPDF$$P50$$Gproquest$$Hfree_for_read</linktopdf><linktohtml>$$Uhttps://www.proquest.com/docview/2624115249?pq-origsite=primo$$EHTML$$P50$$Gproquest$$Hfree_for_read</linktohtml><link.rule.ids>230,314,727,780,784,885,25753,27924,27925,37012,37013,44590,53791,53793,74998</link.rule.ids><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/34951588$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><creatorcontrib>Lamers, Wout S</creatorcontrib><creatorcontrib>Boyack, Kevin</creatorcontrib><creatorcontrib>Larivière, Vincent</creatorcontrib><creatorcontrib>Sugimoto, Cassidy R</creatorcontrib><creatorcontrib>van Eck, Nees Jan</creatorcontrib><creatorcontrib>Waltman, Ludo</creatorcontrib><creatorcontrib>Murray, Dakota</creatorcontrib><title>Investigating disagreement in the scientific literature</title><title>eLife</title><addtitle>Elife</addtitle><description>Disagreement is essential to scientific progress but the extent of disagreement in science, its evolution over time, and the fields in which it happens remain poorly understood. Here we report the development of an approach based on cue phrases that can identify instances of disagreement in scientific articles. These instances are sentences in an article that cite other articles. Applying this approach to a collection of more than four million English-language articles published between 2000 and 2015 period, we determine the level of disagreement in five broad fields within the scientific literature (biomedical and health sciences; life and earth sciences; mathematics and computer science; physical sciences and engineering; and social sciences and humanities) and 817 meso-level fields. Overall, the level of disagreement is highest in the social sciences and humanities, and lowest in mathematics and computer science. However, there is considerable heterogeneity across the meso-level fields, revealing the importance of local disciplinary cultures and the epistemic characteristics of disagreement. Analysis at the level of individual articles reveals notable episodes of disagreement in science, and illustrates how methodological artifacts can confound analyses of scientific texts.</description><subject>Bibliographic coupling</subject><subject>Bibliometrics</subject><subject>citation analysis</subject><subject>Citations</subject><subject>disagreement</subject><subject>Gravitational waves</subject><subject>Interprofessional Relations</subject><subject>Machine learning</subject><subject>Meta-Research</subject><subject>metascience</subject><subject>Natural Language Processing</subject><subject>Natural Science Disciplines</subject><subject>Science</subject><subject>Social Sciences</subject><issn>2050-084X</issn><issn>2050-084X</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2021</creationdate><recordtype>article</recordtype><sourceid>PIMPY</sourceid><sourceid>DOA</sourceid><recordid>eNpdkc1rHCEYh6W0NCHJqfcy0EuhbOLH66iXQglNsrDQSwu9iaOvE5fZmVRnAvnva3eTkFQQvx4eX_0R8oHRcyUlXOAmRTxXXAn1hhxzKumKavj99sX8iJyVsqW1KdCamffkSICRTGp9TNR6vMcyp97NaeybkIrrM-IOx7lJYzPfYlN8qqsUk2-GNGN285LxlLyLbih49jiekF9X339e3qw2P67Xl982Kw-yVSvJI9_3oJT2GCLTPAKw0EamuA68Yx6go9x5aFutIggpmITOGd1Kx8UJWR-8YXJbe5fTzuUHO7lk9xtT7q3Lc_IDWheMYvUKTTsJrPOmEyCdgYCx1UbQ6vp6cN0t3Q6Dr6_KbnglfX0yplvbT_dWK2qkaqvg86MgT3-W-m12l4rHYXAjTkuxvGXAuRQCKvrpP3Q7LXmsX1UpDoxJDqZSXw6Uz1MpGeNzMYzaf_nafb52n2-lP76s_5l9SlP8Bc2YoC8</recordid><startdate>20211224</startdate><enddate>20211224</enddate><creator>Lamers, Wout S</creator><creator>Boyack, Kevin</creator><creator>Larivière, Vincent</creator><creator>Sugimoto, Cassidy R</creator><creator>van Eck, Nees Jan</creator><creator>Waltman, Ludo</creator><creator>Murray, Dakota</creator><general>eLife Sciences Publications Ltd</general><general>eLife Sciences Publications, Ltd</general><scope>CGR</scope><scope>CUY</scope><scope>CVF</scope><scope>ECM</scope><scope>EIF</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>3V.</scope><scope>7X7</scope><scope>7XB</scope><scope>88E</scope><scope>88I</scope><scope>8FE</scope><scope>8FH</scope><scope>8FI</scope><scope>8FJ</scope><scope>8FK</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>AZQEC</scope><scope>BBNVY</scope><scope>BENPR</scope><scope>BHPHI</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>FYUFA</scope><scope>GHDGH</scope><scope>GNUQQ</scope><scope>HCIFZ</scope><scope>K9.</scope><scope>LK8</scope><scope>M0S</scope><scope>M1P</scope><scope>M2P</scope><scope>M7P</scope><scope>PIMPY</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>Q9U</scope><scope>7X8</scope><scope>5PM</scope><scope>DOA</scope><orcidid>https://orcid.org/0000-0001-7814-8951</orcidid><orcidid>https://orcid.org/0000-0001-7176-9579</orcidid><orcidid>https://orcid.org/0000-0001-8448-4521</orcidid><orcidid>https://orcid.org/0000-0001-8249-1752</orcidid><orcidid>https://orcid.org/0000-0002-7119-0169</orcidid><orcidid>https://orcid.org/0000-0002-2733-0689</orcidid></search><sort><creationdate>20211224</creationdate><title>Investigating disagreement in the scientific literature</title><author>Lamers, Wout S ; Boyack, Kevin ; Larivière, Vincent ; Sugimoto, Cassidy R ; van Eck, Nees Jan ; Waltman, Ludo ; Murray, Dakota</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c4567-52f252f25d778cedf182f441d6f1728d2b1c44b02ac46687f4353154ba9865a23</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2021</creationdate><topic>Bibliographic coupling</topic><topic>Bibliometrics</topic><topic>citation analysis</topic><topic>Citations</topic><topic>disagreement</topic><topic>Gravitational waves</topic><topic>Interprofessional Relations</topic><topic>Machine learning</topic><topic>Meta-Research</topic><topic>metascience</topic><topic>Natural Language Processing</topic><topic>Natural Science Disciplines</topic><topic>Science</topic><topic>Social Sciences</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Lamers, Wout S</creatorcontrib><creatorcontrib>Boyack, Kevin</creatorcontrib><creatorcontrib>Larivière, Vincent</creatorcontrib><creatorcontrib>Sugimoto, Cassidy R</creatorcontrib><creatorcontrib>van Eck, Nees Jan</creatorcontrib><creatorcontrib>Waltman, Ludo</creatorcontrib><creatorcontrib>Murray, Dakota</creatorcontrib><collection>Medline</collection><collection>MEDLINE</collection><collection>MEDLINE (Ovid)</collection><collection>MEDLINE</collection><collection>MEDLINE</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>ProQuest Central (Corporate)</collection><collection>ProQuest Health &amp; Medical Collection</collection><collection>ProQuest Central (purchase pre-March 2016)</collection><collection>Medical Database (Alumni Edition)</collection><collection>Science Database (Alumni Edition)</collection><collection>ProQuest SciTech Collection</collection><collection>ProQuest Natural Science Collection</collection><collection>Hospital Premium Collection</collection><collection>Hospital Premium Collection (Alumni Edition)</collection><collection>ProQuest Central (Alumni) (purchase pre-March 2016)</collection><collection>ProQuest Central (Alumni)</collection><collection>ProQuest Central</collection><collection>ProQuest Central Essentials</collection><collection>Biological Science Collection</collection><collection>AUTh Library subscriptions: ProQuest Central</collection><collection>ProQuest Natural Science Collection</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central</collection><collection>Health Research Premium Collection</collection><collection>Health Research Premium Collection (Alumni)</collection><collection>ProQuest Central Student</collection><collection>SciTech Premium Collection (Proquest) (PQ_SDU_P3)</collection><collection>ProQuest Health &amp; Medical Complete (Alumni)</collection><collection>Biological Sciences</collection><collection>Health &amp; Medical Collection (Alumni Edition)</collection><collection>Medical Database</collection><collection>ProQuest Science Journals</collection><collection>Biological Science Database</collection><collection>Publicly Available Content (ProQuest)</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>ProQuest Central Basic</collection><collection>MEDLINE - Academic</collection><collection>PubMed Central (Full Participant titles)</collection><collection>Open Access: DOAJ - Directory of Open Access Journals</collection><jtitle>eLife</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Lamers, Wout S</au><au>Boyack, Kevin</au><au>Larivière, Vincent</au><au>Sugimoto, Cassidy R</au><au>van Eck, Nees Jan</au><au>Waltman, Ludo</au><au>Murray, Dakota</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Investigating disagreement in the scientific literature</atitle><jtitle>eLife</jtitle><addtitle>Elife</addtitle><date>2021-12-24</date><risdate>2021</risdate><volume>10</volume><issn>2050-084X</issn><eissn>2050-084X</eissn><abstract>Disagreement is essential to scientific progress but the extent of disagreement in science, its evolution over time, and the fields in which it happens remain poorly understood. Here we report the development of an approach based on cue phrases that can identify instances of disagreement in scientific articles. These instances are sentences in an article that cite other articles. Applying this approach to a collection of more than four million English-language articles published between 2000 and 2015 period, we determine the level of disagreement in five broad fields within the scientific literature (biomedical and health sciences; life and earth sciences; mathematics and computer science; physical sciences and engineering; and social sciences and humanities) and 817 meso-level fields. Overall, the level of disagreement is highest in the social sciences and humanities, and lowest in mathematics and computer science. However, there is considerable heterogeneity across the meso-level fields, revealing the importance of local disciplinary cultures and the epistemic characteristics of disagreement. Analysis at the level of individual articles reveals notable episodes of disagreement in science, and illustrates how methodological artifacts can confound analyses of scientific texts.</abstract><cop>England</cop><pub>eLife Sciences Publications Ltd</pub><pmid>34951588</pmid><doi>10.7554/eLife.72737</doi><orcidid>https://orcid.org/0000-0001-7814-8951</orcidid><orcidid>https://orcid.org/0000-0001-7176-9579</orcidid><orcidid>https://orcid.org/0000-0001-8448-4521</orcidid><orcidid>https://orcid.org/0000-0001-8249-1752</orcidid><orcidid>https://orcid.org/0000-0002-7119-0169</orcidid><orcidid>https://orcid.org/0000-0002-2733-0689</orcidid><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier ISSN: 2050-084X
ispartof eLife, 2021-12, Vol.10
issn 2050-084X
2050-084X
language eng
recordid cdi_doaj_primary_oai_doaj_org_article_ad971edf80b541bc9b345a94def68930
source Open Access: PubMed Central; Publicly Available Content (ProQuest)
subjects Bibliographic coupling
Bibliometrics
citation analysis
Citations
disagreement
Gravitational waves
Interprofessional Relations
Machine learning
Meta-Research
metascience
Natural Language Processing
Natural Science Disciplines
Science
Social Sciences
title Investigating disagreement in the scientific literature
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-07T22%3A40%3A11IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_doaj_&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Investigating%20disagreement%20in%20the%20scientific%20literature&rft.jtitle=eLife&rft.au=Lamers,%20Wout%20S&rft.date=2021-12-24&rft.volume=10&rft.issn=2050-084X&rft.eissn=2050-084X&rft_id=info:doi/10.7554/eLife.72737&rft_dat=%3Cproquest_doaj_%3E2614225334%3C/proquest_doaj_%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-c4567-52f252f25d778cedf182f441d6f1728d2b1c44b02ac46687f4353154ba9865a23%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_pqid=2624115249&rft_id=info:pmid/34951588&rfr_iscdi=true