Loading…

Feature blindness: A challenge for understanding and modelling visual object recognition

Humans rely heavily on the shape of objects to recognise them. Recently, it has been argued that Convolutional Neural Networks (CNNs) can also show a shape-bias, provided their learning environment contains this bias. This has led to the proposal that CNNs provide good mechanistic models of shape-bi...

Full description

Saved in:
Bibliographic Details
Published in:PLoS computational biology 2022-05, Vol.18 (5), p.e1009572-e1009572
Main Authors: Malhotra, Gaurav, Dujmović, Marin, Bowers, Jeffrey S
Format: Article
Language:English
Subjects:
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by cdi_FETCH-LOGICAL-c633t-c88f8de3dbd94f66f63f3de942b0e075b59cd0dbb885809ce630d47f73bf93323
cites cdi_FETCH-LOGICAL-c633t-c88f8de3dbd94f66f63f3de942b0e075b59cd0dbb885809ce630d47f73bf93323
container_end_page e1009572
container_issue 5
container_start_page e1009572
container_title PLoS computational biology
container_volume 18
creator Malhotra, Gaurav
Dujmović, Marin
Bowers, Jeffrey S
description Humans rely heavily on the shape of objects to recognise them. Recently, it has been argued that Convolutional Neural Networks (CNNs) can also show a shape-bias, provided their learning environment contains this bias. This has led to the proposal that CNNs provide good mechanistic models of shape-bias and, more generally, human visual processing. However, it is also possible that humans and CNNs show a shape-bias for very different reasons, namely, shape-bias in humans may be a consequence of architectural and cognitive constraints whereas CNNs show a shape-bias as a consequence of learning the statistics of the environment. We investigated this question by exploring shape-bias in humans and CNNs when they learn in a novel environment. We observed that, in this new environment, humans (i) focused on shape and overlooked many non-shape features, even when non-shape features were more diagnostic, (ii) learned based on only one out of multiple predictive features, and (iii) failed to learn when global features, such as shape, were absent. This behaviour contrasted with the predictions of a statistical inference model with no priors, showing the strong role that shape-bias plays in human feature selection. It also contrasted with CNNs that (i) preferred to categorise objects based on non-shape features, and (ii) increased reliance on these non-shape features as they became more predictive. This was the case even when the CNN was pre-trained to have a shape-bias and the convolutional backbone was frozen. These results suggest that shape-bias has a different source in humans and CNNs: while learning in CNNs is driven by the statistical properties of the environment, humans are highly constrained by their previous biases, which suggests that cognitive constraints play a key role in how humans learn to recognise novel objects.
doi_str_mv 10.1371/journal.pcbi.1009572
format article
fullrecord <record><control><sourceid>gale_plos_</sourceid><recordid>TN_cdi_plos_journals_2677645404</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><galeid>A705774439</galeid><doaj_id>oai_doaj_org_article_8158b99204a148cc8dc35f7602a21242</doaj_id><sourcerecordid>A705774439</sourcerecordid><originalsourceid>FETCH-LOGICAL-c633t-c88f8de3dbd94f66f63f3de942b0e075b59cd0dbb885809ce630d47f73bf93323</originalsourceid><addsrcrecordid>eNqVkkuLFDEQgBtR3HX1H4g2eNHDjOnOsz0Iw-LqwKLgA7yFdFLpzZBJxqR70X9v2plddsSL5FCVqq-eVFU9bdCywbx5vYlTCsovd7p3ywahjvL2XnXaUIoXHFNx_45-Uj3KeYNQUTv2sDrBlDJUvKfV9wtQ45Sg7r0LJkDOb-pVra-U9xAGqG1M9RQMpDyqYFwY6iLqbTTg_fy7dnlSvo79BvRYJ9BxCG50MTyuHljlMzw5yLPq28W7r-cfFpef3q_PV5cLzTAeF1oIKwxg05uOWMYswxYb6EjbI0Cc9rTTBpm-F4IK1GlgGBnCLce97TBu8Vn1fJ9352OWh6Vk2TLOGaEEkUKs94SJaiN3yW1V-iWjcvKPIaZBqjQ67UGKhoq-61pEVEOE1sJoTC1nqFVt05K52ttDtanfgtEQxqT8UdJjT3BXcojXsmtKry0uCV4eEqT4Y4I8yq3LuixTBYjT3DcjAmHGZvTFX-i_p1vuqUGVAVywsdTV5RnYOh0DWFfsK44o54TgrgS8OgoozAg_x0FNOcv1l8__wX48Zsme1SnmnMDebqVBcr7Ym_blfLHycLEl7Nndjd4G3Zwo_g2LGeey</addsrcrecordid><sourcetype>Open Website</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2677645404</pqid></control><display><type>article</type><title>Feature blindness: A challenge for understanding and modelling visual object recognition</title><source>PubMed (Medline)</source><source>Publicly Available Content (ProQuest)</source><creator>Malhotra, Gaurav ; Dujmović, Marin ; Bowers, Jeffrey S</creator><contributor>Richards, Blake A</contributor><creatorcontrib>Malhotra, Gaurav ; Dujmović, Marin ; Bowers, Jeffrey S ; Richards, Blake A</creatorcontrib><description>Humans rely heavily on the shape of objects to recognise them. Recently, it has been argued that Convolutional Neural Networks (CNNs) can also show a shape-bias, provided their learning environment contains this bias. This has led to the proposal that CNNs provide good mechanistic models of shape-bias and, more generally, human visual processing. However, it is also possible that humans and CNNs show a shape-bias for very different reasons, namely, shape-bias in humans may be a consequence of architectural and cognitive constraints whereas CNNs show a shape-bias as a consequence of learning the statistics of the environment. We investigated this question by exploring shape-bias in humans and CNNs when they learn in a novel environment. We observed that, in this new environment, humans (i) focused on shape and overlooked many non-shape features, even when non-shape features were more diagnostic, (ii) learned based on only one out of multiple predictive features, and (iii) failed to learn when global features, such as shape, were absent. This behaviour contrasted with the predictions of a statistical inference model with no priors, showing the strong role that shape-bias plays in human feature selection. It also contrasted with CNNs that (i) preferred to categorise objects based on non-shape features, and (ii) increased reliance on these non-shape features as they became more predictive. This was the case even when the CNN was pre-trained to have a shape-bias and the convolutional backbone was frozen. These results suggest that shape-bias has a different source in humans and CNNs: while learning in CNNs is driven by the statistical properties of the environment, humans are highly constrained by their previous biases, which suggests that cognitive constraints play a key role in how humans learn to recognise novel objects.</description><identifier>ISSN: 1553-7358</identifier><identifier>ISSN: 1553-734X</identifier><identifier>EISSN: 1553-7358</identifier><identifier>DOI: 10.1371/journal.pcbi.1009572</identifier><identifier>PMID: 35560155</identifier><language>eng</language><publisher>United States: Public Library of Science</publisher><subject>Artificial neural networks ; Bias ; Biology and Life Sciences ; Cognitive ability ; Computer and Information Sciences ; Constraints ; Environmental statistics ; Human performance ; Hypotheses ; Information processing ; Learning ; Mathematical models ; Neural networks ; Object recognition ; Pattern recognition ; Physical Sciences ; Research and Analysis Methods ; Shape recognition ; Social Sciences ; Statistical inference ; Visual perception</subject><ispartof>PLoS computational biology, 2022-05, Vol.18 (5), p.e1009572-e1009572</ispartof><rights>COPYRIGHT 2022 Public Library of Science</rights><rights>2022 Malhotra et al. This is an open access article distributed under the terms of the Creative Commons Attribution License: http://creativecommons.org/licenses/by/4.0/ (the “License”), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><rights>2022 Malhotra et al 2022 Malhotra et al</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c633t-c88f8de3dbd94f66f63f3de942b0e075b59cd0dbb885809ce630d47f73bf93323</citedby><cites>FETCH-LOGICAL-c633t-c88f8de3dbd94f66f63f3de942b0e075b59cd0dbb885809ce630d47f73bf93323</cites><orcidid>0000-0003-1523-227X ; 0000-0002-6868-9655</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://www.proquest.com/docview/2677645404/fulltextPDF?pq-origsite=primo$$EPDF$$P50$$Gproquest$$Hfree_for_read</linktopdf><linktohtml>$$Uhttps://www.proquest.com/docview/2677645404?pq-origsite=primo$$EHTML$$P50$$Gproquest$$Hfree_for_read</linktohtml><link.rule.ids>230,314,727,780,784,885,25753,27924,27925,37012,37013,44590,53791,53793,75126</link.rule.ids><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/35560155$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><contributor>Richards, Blake A</contributor><creatorcontrib>Malhotra, Gaurav</creatorcontrib><creatorcontrib>Dujmović, Marin</creatorcontrib><creatorcontrib>Bowers, Jeffrey S</creatorcontrib><title>Feature blindness: A challenge for understanding and modelling visual object recognition</title><title>PLoS computational biology</title><addtitle>PLoS Comput Biol</addtitle><description>Humans rely heavily on the shape of objects to recognise them. Recently, it has been argued that Convolutional Neural Networks (CNNs) can also show a shape-bias, provided their learning environment contains this bias. This has led to the proposal that CNNs provide good mechanistic models of shape-bias and, more generally, human visual processing. However, it is also possible that humans and CNNs show a shape-bias for very different reasons, namely, shape-bias in humans may be a consequence of architectural and cognitive constraints whereas CNNs show a shape-bias as a consequence of learning the statistics of the environment. We investigated this question by exploring shape-bias in humans and CNNs when they learn in a novel environment. We observed that, in this new environment, humans (i) focused on shape and overlooked many non-shape features, even when non-shape features were more diagnostic, (ii) learned based on only one out of multiple predictive features, and (iii) failed to learn when global features, such as shape, were absent. This behaviour contrasted with the predictions of a statistical inference model with no priors, showing the strong role that shape-bias plays in human feature selection. It also contrasted with CNNs that (i) preferred to categorise objects based on non-shape features, and (ii) increased reliance on these non-shape features as they became more predictive. This was the case even when the CNN was pre-trained to have a shape-bias and the convolutional backbone was frozen. These results suggest that shape-bias has a different source in humans and CNNs: while learning in CNNs is driven by the statistical properties of the environment, humans are highly constrained by their previous biases, which suggests that cognitive constraints play a key role in how humans learn to recognise novel objects.</description><subject>Artificial neural networks</subject><subject>Bias</subject><subject>Biology and Life Sciences</subject><subject>Cognitive ability</subject><subject>Computer and Information Sciences</subject><subject>Constraints</subject><subject>Environmental statistics</subject><subject>Human performance</subject><subject>Hypotheses</subject><subject>Information processing</subject><subject>Learning</subject><subject>Mathematical models</subject><subject>Neural networks</subject><subject>Object recognition</subject><subject>Pattern recognition</subject><subject>Physical Sciences</subject><subject>Research and Analysis Methods</subject><subject>Shape recognition</subject><subject>Social Sciences</subject><subject>Statistical inference</subject><subject>Visual perception</subject><issn>1553-7358</issn><issn>1553-734X</issn><issn>1553-7358</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2022</creationdate><recordtype>article</recordtype><sourceid>PIMPY</sourceid><sourceid>DOA</sourceid><recordid>eNqVkkuLFDEQgBtR3HX1H4g2eNHDjOnOsz0Iw-LqwKLgA7yFdFLpzZBJxqR70X9v2plddsSL5FCVqq-eVFU9bdCywbx5vYlTCsovd7p3ywahjvL2XnXaUIoXHFNx_45-Uj3KeYNQUTv2sDrBlDJUvKfV9wtQ45Sg7r0LJkDOb-pVra-U9xAGqG1M9RQMpDyqYFwY6iLqbTTg_fy7dnlSvo79BvRYJ9BxCG50MTyuHljlMzw5yLPq28W7r-cfFpef3q_PV5cLzTAeF1oIKwxg05uOWMYswxYb6EjbI0Cc9rTTBpm-F4IK1GlgGBnCLce97TBu8Vn1fJ9352OWh6Vk2TLOGaEEkUKs94SJaiN3yW1V-iWjcvKPIaZBqjQ67UGKhoq-61pEVEOE1sJoTC1nqFVt05K52ttDtanfgtEQxqT8UdJjT3BXcojXsmtKry0uCV4eEqT4Y4I8yq3LuixTBYjT3DcjAmHGZvTFX-i_p1vuqUGVAVywsdTV5RnYOh0DWFfsK44o54TgrgS8OgoozAg_x0FNOcv1l8__wX48Zsme1SnmnMDebqVBcr7Ym_blfLHycLEl7Nndjd4G3Zwo_g2LGeey</recordid><startdate>20220501</startdate><enddate>20220501</enddate><creator>Malhotra, Gaurav</creator><creator>Dujmović, Marin</creator><creator>Bowers, Jeffrey S</creator><general>Public Library of Science</general><general>Public Library of Science (PLoS)</general><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>ISN</scope><scope>ISR</scope><scope>3V.</scope><scope>7QO</scope><scope>7QP</scope><scope>7TK</scope><scope>7TM</scope><scope>7X7</scope><scope>7XB</scope><scope>88E</scope><scope>8AL</scope><scope>8FD</scope><scope>8FE</scope><scope>8FG</scope><scope>8FH</scope><scope>8FI</scope><scope>8FJ</scope><scope>8FK</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>ARAPS</scope><scope>AZQEC</scope><scope>BBNVY</scope><scope>BENPR</scope><scope>BGLVJ</scope><scope>BHPHI</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>FR3</scope><scope>FYUFA</scope><scope>GHDGH</scope><scope>GNUQQ</scope><scope>HCIFZ</scope><scope>JQ2</scope><scope>K7-</scope><scope>K9.</scope><scope>LK8</scope><scope>M0N</scope><scope>M0S</scope><scope>M1P</scope><scope>M7P</scope><scope>P5Z</scope><scope>P62</scope><scope>P64</scope><scope>PIMPY</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>Q9U</scope><scope>RC3</scope><scope>7X8</scope><scope>5PM</scope><scope>DOA</scope><orcidid>https://orcid.org/0000-0003-1523-227X</orcidid><orcidid>https://orcid.org/0000-0002-6868-9655</orcidid></search><sort><creationdate>20220501</creationdate><title>Feature blindness: A challenge for understanding and modelling visual object recognition</title><author>Malhotra, Gaurav ; Dujmović, Marin ; Bowers, Jeffrey S</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c633t-c88f8de3dbd94f66f63f3de942b0e075b59cd0dbb885809ce630d47f73bf93323</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2022</creationdate><topic>Artificial neural networks</topic><topic>Bias</topic><topic>Biology and Life Sciences</topic><topic>Cognitive ability</topic><topic>Computer and Information Sciences</topic><topic>Constraints</topic><topic>Environmental statistics</topic><topic>Human performance</topic><topic>Hypotheses</topic><topic>Information processing</topic><topic>Learning</topic><topic>Mathematical models</topic><topic>Neural networks</topic><topic>Object recognition</topic><topic>Pattern recognition</topic><topic>Physical Sciences</topic><topic>Research and Analysis Methods</topic><topic>Shape recognition</topic><topic>Social Sciences</topic><topic>Statistical inference</topic><topic>Visual perception</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Malhotra, Gaurav</creatorcontrib><creatorcontrib>Dujmović, Marin</creatorcontrib><creatorcontrib>Bowers, Jeffrey S</creatorcontrib><collection>PubMed</collection><collection>CrossRef</collection><collection>Gale In Context: Canada</collection><collection>Science in Context</collection><collection>ProQuest Central (Corporate)</collection><collection>Biotechnology Research Abstracts</collection><collection>Calcium &amp; Calcified Tissue Abstracts</collection><collection>Neurosciences Abstracts</collection><collection>Nucleic Acids Abstracts</collection><collection>ProQuest_Health &amp; Medical Collection</collection><collection>ProQuest Central (purchase pre-March 2016)</collection><collection>Medical Database (Alumni Edition)</collection><collection>Computing Database (Alumni Edition)</collection><collection>Technology Research Database</collection><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>ProQuest Natural Science Collection</collection><collection>Hospital Premium Collection</collection><collection>Hospital Premium Collection (Alumni Edition)</collection><collection>ProQuest Central (Alumni) (purchase pre-March 2016)</collection><collection>ProQuest Central (Alumni)</collection><collection>ProQuest Central UK/Ireland</collection><collection>Advanced Technologies &amp; Aerospace Collection</collection><collection>ProQuest Central Essentials</collection><collection>Biological Science Collection</collection><collection>ProQuest Central</collection><collection>Technology Collection</collection><collection>ProQuest Natural Science Collection</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central</collection><collection>Engineering Research Database</collection><collection>Health Research Premium Collection</collection><collection>Health Research Premium Collection (Alumni)</collection><collection>ProQuest Central Student</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Computer Science Collection</collection><collection>Computer Science Database</collection><collection>ProQuest Health &amp; Medical Complete (Alumni)</collection><collection>ProQuest Biological Science Collection</collection><collection>Computing Database</collection><collection>Health &amp; Medical Collection (Alumni Edition)</collection><collection>PML(ProQuest Medical Library)</collection><collection>Biological Science Database</collection><collection>ProQuest advanced technologies &amp; aerospace journals</collection><collection>ProQuest Advanced Technologies &amp; Aerospace Collection</collection><collection>Biotechnology and BioEngineering Abstracts</collection><collection>Publicly Available Content (ProQuest)</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>ProQuest Central Basic</collection><collection>Genetics Abstracts</collection><collection>MEDLINE - Academic</collection><collection>PubMed Central (Full Participant titles)</collection><collection>DOAJ Directory of Open Access Journals</collection><jtitle>PLoS computational biology</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Malhotra, Gaurav</au><au>Dujmović, Marin</au><au>Bowers, Jeffrey S</au><au>Richards, Blake A</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Feature blindness: A challenge for understanding and modelling visual object recognition</atitle><jtitle>PLoS computational biology</jtitle><addtitle>PLoS Comput Biol</addtitle><date>2022-05-01</date><risdate>2022</risdate><volume>18</volume><issue>5</issue><spage>e1009572</spage><epage>e1009572</epage><pages>e1009572-e1009572</pages><issn>1553-7358</issn><issn>1553-734X</issn><eissn>1553-7358</eissn><abstract>Humans rely heavily on the shape of objects to recognise them. Recently, it has been argued that Convolutional Neural Networks (CNNs) can also show a shape-bias, provided their learning environment contains this bias. This has led to the proposal that CNNs provide good mechanistic models of shape-bias and, more generally, human visual processing. However, it is also possible that humans and CNNs show a shape-bias for very different reasons, namely, shape-bias in humans may be a consequence of architectural and cognitive constraints whereas CNNs show a shape-bias as a consequence of learning the statistics of the environment. We investigated this question by exploring shape-bias in humans and CNNs when they learn in a novel environment. We observed that, in this new environment, humans (i) focused on shape and overlooked many non-shape features, even when non-shape features were more diagnostic, (ii) learned based on only one out of multiple predictive features, and (iii) failed to learn when global features, such as shape, were absent. This behaviour contrasted with the predictions of a statistical inference model with no priors, showing the strong role that shape-bias plays in human feature selection. It also contrasted with CNNs that (i) preferred to categorise objects based on non-shape features, and (ii) increased reliance on these non-shape features as they became more predictive. This was the case even when the CNN was pre-trained to have a shape-bias and the convolutional backbone was frozen. These results suggest that shape-bias has a different source in humans and CNNs: while learning in CNNs is driven by the statistical properties of the environment, humans are highly constrained by their previous biases, which suggests that cognitive constraints play a key role in how humans learn to recognise novel objects.</abstract><cop>United States</cop><pub>Public Library of Science</pub><pmid>35560155</pmid><doi>10.1371/journal.pcbi.1009572</doi><orcidid>https://orcid.org/0000-0003-1523-227X</orcidid><orcidid>https://orcid.org/0000-0002-6868-9655</orcidid><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier ISSN: 1553-7358
ispartof PLoS computational biology, 2022-05, Vol.18 (5), p.e1009572-e1009572
issn 1553-7358
1553-734X
1553-7358
language eng
recordid cdi_plos_journals_2677645404
source PubMed (Medline); Publicly Available Content (ProQuest)
subjects Artificial neural networks
Bias
Biology and Life Sciences
Cognitive ability
Computer and Information Sciences
Constraints
Environmental statistics
Human performance
Hypotheses
Information processing
Learning
Mathematical models
Neural networks
Object recognition
Pattern recognition
Physical Sciences
Research and Analysis Methods
Shape recognition
Social Sciences
Statistical inference
Visual perception
title Feature blindness: A challenge for understanding and modelling visual object recognition
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-24T20%3A42%3A17IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-gale_plos_&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Feature%20blindness:%20A%20challenge%20for%20understanding%20and%20modelling%20visual%20object%20recognition&rft.jtitle=PLoS%20computational%20biology&rft.au=Malhotra,%20Gaurav&rft.date=2022-05-01&rft.volume=18&rft.issue=5&rft.spage=e1009572&rft.epage=e1009572&rft.pages=e1009572-e1009572&rft.issn=1553-7358&rft.eissn=1553-7358&rft_id=info:doi/10.1371/journal.pcbi.1009572&rft_dat=%3Cgale_plos_%3EA705774439%3C/gale_plos_%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-c633t-c88f8de3dbd94f66f63f3de942b0e075b59cd0dbb885809ce630d47f73bf93323%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_pqid=2677645404&rft_id=info:pmid/35560155&rft_galeid=A705774439&rfr_iscdi=true