Compositional Processing Emerges in Neural Networks Solving Math Problems

A longstanding question in cognitive science concerns the learning mechanisms underlying compositionality in human cognition. Humans can infer the structured relationships (e.g., grammatical rules) implicit in their sensory observations (e.g., auditory speech), and use this knowledge to guide the co...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2021-05
Main Authors: Russin, Jacob, Fernandez, Roland, Palangi, Hamid, Rosen, Eric, Jojic, Nebojsa, Smolensky, Paul, Gao, Jianfeng
Format: Article
Language:English
Subjects:
Artificial neural networks
Cognition
Composition
Neural networks
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by
cites
container_end_page
container_issue
container_start_page
container_title arXiv.org
container_volume
creator Russin, Jacob
Fernandez, Roland
Palangi, Hamid
Rosen, Eric
Jojic, Nebojsa
Smolensky, Paul
Gao, Jianfeng
description A longstanding question in cognitive science concerns the learning mechanisms underlying compositionality in human cognition. Humans can infer the structured relationships (e.g., grammatical rules) implicit in their sensory observations (e.g., auditory speech), and use this knowledge to guide the composition of simpler meanings into complex wholes. Recent progress in artificial neural networks has shown that when large models are trained on enough linguistic data, grammatical structure emerges in their representations. We extend this work to the domain of mathematical reasoning, where it is possible to formulate precise hypotheses about how meanings (e.g., the quantities corresponding to numerals) should be composed according to structured rules (e.g., order of operations). Our work shows that neural networks are not only able to infer something about the structured relationships implicit in their training data, but can also deploy this knowledge to guide the composition of individual meanings into composite wholes.
format article
fullrecord <record><control><sourceid>proquest</sourceid><recordid>TN_cdi_proquest_journals_2529616101</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2529616101</sourcerecordid><originalsourceid>FETCH-proquest_journals_25296161013</originalsourceid><addsrcrecordid>eNqNjc0KgkAURocgSMp3GGgtzE9arcWoRRLUXixuNjZ6be5Yr59CD9DqLL7zcSYsUFrLaLNSasZColoIoZK1imMdsEOKTYdkvMG2tPzk8AZEpq141oCrgLhpeQ69G8Yc_Afdk_gZ7XtUjqV_jJerhYYWbHovLUH445wtd9kl3Uedw1cP5IsaezdEqFCx2iYykULq_6wvtws8oQ</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2529616101</pqid></control><display><type>article</type><title>Compositional Processing Emerges in Neural Networks Solving Math Problems</title><source>Publicly Available Content Database</source><creator>Russin, Jacob ; Fernandez, Roland ; Palangi, Hamid ; Rosen, Eric ; Jojic, Nebojsa ; Smolensky, Paul ; Gao, Jianfeng</creator><creatorcontrib>Russin, Jacob ; Fernandez, Roland ; Palangi, Hamid ; Rosen, Eric ; Jojic, Nebojsa ; Smolensky, Paul ; Gao, Jianfeng</creatorcontrib><description>A longstanding question in cognitive science concerns the learning mechanisms underlying compositionality in human cognition. Humans can infer the structured relationships (e.g., grammatical rules) implicit in their sensory observations (e.g., auditory speech), and use this knowledge to guide the composition of simpler meanings into complex wholes. Recent progress in artificial neural networks has shown that when large models are trained on enough linguistic data, grammatical structure emerges in their representations. We extend this work to the domain of mathematical reasoning, where it is possible to formulate precise hypotheses about how meanings (e.g., the quantities corresponding to numerals) should be composed according to structured rules (e.g., order of operations). Our work shows that neural networks are not only able to infer something about the structured relationships implicit in their training data, but can also deploy this knowledge to guide the composition of individual meanings into composite wholes.</description><identifier>EISSN: 2331-8422</identifier><language>eng</language><publisher>Ithaca: Cornell University Library, arXiv.org</publisher><subject>Artificial neural networks ; Cognition ; Composition ; Neural networks</subject><ispartof>arXiv.org, 2021-05</ispartof><rights>2021. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://www.proquest.com/docview/2529616101?pq-origsite=primo$$EHTML$$P50$$Gproquest$$Hfree_for_read</linktohtml><link.rule.ids>777,781,25734,36993,44571</link.rule.ids></links><search><creatorcontrib>Russin, Jacob</creatorcontrib><creatorcontrib>Fernandez, Roland</creatorcontrib><creatorcontrib>Palangi, Hamid</creatorcontrib><creatorcontrib>Rosen, Eric</creatorcontrib><creatorcontrib>Jojic, Nebojsa</creatorcontrib><creatorcontrib>Smolensky, Paul</creatorcontrib><creatorcontrib>Gao, Jianfeng</creatorcontrib><title>Compositional Processing Emerges in Neural Networks Solving Math Problems</title><title>arXiv.org</title><description>A longstanding question in cognitive science concerns the learning mechanisms underlying compositionality in human cognition. Humans can infer the structured relationships (e.g., grammatical rules) implicit in their sensory observations (e.g., auditory speech), and use this knowledge to guide the composition of simpler meanings into complex wholes. Recent progress in artificial neural networks has shown that when large models are trained on enough linguistic data, grammatical structure emerges in their representations. We extend this work to the domain of mathematical reasoning, where it is possible to formulate precise hypotheses about how meanings (e.g., the quantities corresponding to numerals) should be composed according to structured rules (e.g., order of operations). Our work shows that neural networks are not only able to infer something about the structured relationships implicit in their training data, but can also deploy this knowledge to guide the composition of individual meanings into composite wholes.</description><subject>Artificial neural networks</subject><subject>Cognition</subject><subject>Composition</subject><subject>Neural networks</subject><issn>2331-8422</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2021</creationdate><recordtype>article</recordtype><sourceid>PIMPY</sourceid><recordid>eNqNjc0KgkAURocgSMp3GGgtzE9arcWoRRLUXixuNjZ6be5Yr59CD9DqLL7zcSYsUFrLaLNSasZColoIoZK1imMdsEOKTYdkvMG2tPzk8AZEpq141oCrgLhpeQ69G8Yc_Afdk_gZ7XtUjqV_jJerhYYWbHovLUH445wtd9kl3Uedw1cP5IsaezdEqFCx2iYykULq_6wvtws8oQ</recordid><startdate>20210519</startdate><enddate>20210519</enddate><creator>Russin, Jacob</creator><creator>Fernandez, Roland</creator><creator>Palangi, Hamid</creator><creator>Rosen, Eric</creator><creator>Jojic, Nebojsa</creator><creator>Smolensky, Paul</creator><creator>Gao, Jianfeng</creator><general>Cornell University Library, arXiv.org</general><scope>8FE</scope><scope>8FG</scope><scope>ABJCF</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BGLVJ</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>HCIFZ</scope><scope>L6V</scope><scope>M7S</scope><scope>PIMPY</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>PTHSS</scope></search><sort><creationdate>20210519</creationdate><title>Compositional Processing Emerges in Neural Networks Solving Math Problems</title><author>Russin, Jacob ; Fernandez, Roland ; Palangi, Hamid ; Rosen, Eric ; Jojic, Nebojsa ; Smolensky, Paul ; Gao, Jianfeng</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-proquest_journals_25296161013</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2021</creationdate><topic>Artificial neural networks</topic><topic>Cognition</topic><topic>Composition</topic><topic>Neural networks</topic><toplevel>online_resources</toplevel><creatorcontrib>Russin, Jacob</creatorcontrib><creatorcontrib>Fernandez, Roland</creatorcontrib><creatorcontrib>Palangi, Hamid</creatorcontrib><creatorcontrib>Rosen, Eric</creatorcontrib><creatorcontrib>Jojic, Nebojsa</creatorcontrib><creatorcontrib>Smolensky, Paul</creatorcontrib><creatorcontrib>Gao, Jianfeng</creatorcontrib><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>Materials Science &amp; Engineering Collection</collection><collection>ProQuest Central (Alumni)</collection><collection>ProQuest Central</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>Technology Collection</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Engineering Collection</collection><collection>Engineering Database</collection><collection>Publicly Available Content Database</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>Engineering Collection</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Russin, Jacob</au><au>Fernandez, Roland</au><au>Palangi, Hamid</au><au>Rosen, Eric</au><au>Jojic, Nebojsa</au><au>Smolensky, Paul</au><au>Gao, Jianfeng</au><format>book</format><genre>document</genre><ristype>GEN</ristype><atitle>Compositional Processing Emerges in Neural Networks Solving Math Problems</atitle><jtitle>arXiv.org</jtitle><date>2021-05-19</date><risdate>2021</risdate><eissn>2331-8422</eissn><abstract>A longstanding question in cognitive science concerns the learning mechanisms underlying compositionality in human cognition. Humans can infer the structured relationships (e.g., grammatical rules) implicit in their sensory observations (e.g., auditory speech), and use this knowledge to guide the composition of simpler meanings into complex wholes. Recent progress in artificial neural networks has shown that when large models are trained on enough linguistic data, grammatical structure emerges in their representations. We extend this work to the domain of mathematical reasoning, where it is possible to formulate precise hypotheses about how meanings (e.g., the quantities corresponding to numerals) should be composed according to structured rules (e.g., order of operations). Our work shows that neural networks are not only able to infer something about the structured relationships implicit in their training data, but can also deploy this knowledge to guide the composition of individual meanings into composite wholes.</abstract><cop>Ithaca</cop><pub>Cornell University Library, arXiv.org</pub><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier EISSN: 2331-8422
ispartof arXiv.org, 2021-05
issn 2331-8422
language eng
recordid cdi_proquest_journals_2529616101
source Publicly Available Content Database
subjects Artificial neural networks
Cognition
Composition
Neural networks
title Compositional Processing Emerges in Neural Networks Solving Math Problems
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-20T15%3A01%3A17IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest&rft_val_fmt=info:ofi/fmt:kev:mtx:book&rft.genre=document&rft.atitle=Compositional%20Processing%20Emerges%20in%20Neural%20Networks%20Solving%20Math%20Problems&rft.jtitle=arXiv.org&rft.au=Russin,%20Jacob&rft.date=2021-05-19&rft.eissn=2331-8422&rft_id=info:doi/&rft_dat=%3Cproquest%3E2529616101%3C/proquest%3E%3Cgrp_id%3Ecdi_FETCH-proquest_journals_25296161013%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_pqid=2529616101&rft_id=info:pmid/&rfr_iscdi=true