Loading…

SLR: A million-scale comprehensive crossword dataset for simultaneous learning and reasoning

The progress of the natural language understanding (NLU) community has put forward higher demands for the knowledge reserve and reasoning ability of the model. However, existing schemes for improving model capabilities often split them into two separate tasks. The task of enabling models to learn an...

Full description

Saved in:
Bibliographic Details
Published in:Neurocomputing (Amsterdam) 2023-10, Vol.554, p.126591, Article 126591
Main Authors: Wang, Chao, Zhu, Tinghui, Li, Zhixu, Liu, Jingping
Format: Article
Language:English
Subjects:
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by cdi_FETCH-LOGICAL-c306t-7880dc68c1abf22442d2373923c3782180406abf587d75446194abac85fde5493
cites cdi_FETCH-LOGICAL-c306t-7880dc68c1abf22442d2373923c3782180406abf587d75446194abac85fde5493
container_end_page
container_issue
container_start_page 126591
container_title Neurocomputing (Amsterdam)
container_volume 554
creator Wang, Chao
Zhu, Tinghui
Li, Zhixu
Liu, Jingping
description The progress of the natural language understanding (NLU) community has put forward higher demands for the knowledge reserve and reasoning ability of the model. However, existing schemes for improving model capabilities often split them into two separate tasks. The task of enabling models to learn and reason about knowledge simultaneously has not received sufficient attention. In this paper, we propose a novel crossword-based NLU task that imparts knowledge information to a model by solving crossword clues and simultaneously trains the model to infer new knowledge from existing knowledge. To this end, we construct a comprehensive crossword dataset SLR containing more than 4 million unique clue-answer pairs. Compared to existing crossword datasets, SLR is more comprehensive and contains linguistic knowledge, expertise in various fields, and commonsense knowledge. Meanwhile, to evaluate the reasoning ability of the model, we design clever details in the reasoning of the answers, most clues require the solver to reason through two or more pieces of knowledge to arrive at an answer. We analyze the composition of the dataset and the similarities and differences of the various types of clues via sampling and consider various data partitioning methods to enhance the generalization ability of the training set. Furthermore, we test the performance of several different advanced models and methods on this dataset and analyze the strengths and weaknesses of each. An interesting conclusion is that even powerful language models perform poorly on tasks that require reasoning.
doi_str_mv 10.1016/j.neucom.2023.126591
format article
fullrecord <record><control><sourceid>elsevier_cross</sourceid><recordid>TN_cdi_crossref_primary_10_1016_j_neucom_2023_126591</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><els_id>S0925231223007142</els_id><sourcerecordid>S0925231223007142</sourcerecordid><originalsourceid>FETCH-LOGICAL-c306t-7880dc68c1abf22442d2373923c3782180406abf587d75446194abac85fde5493</originalsourceid><addsrcrecordid>eNp9kNtKxDAURYMoOI7-gQ_5gdbcmqY-CMPgDQYEL29CyCSnmqFNhqQz4t_bsT77dNgc9mKzELqkpKSEyqtNGWBnY18ywnhJmawaeoRmVNWsUEzJYzQjDasKxik7RWc5bwihNWXNDL2_rJ6v8QL3vut8DEW2pgM8srYJPiFkvx9Tijl_xeSwM4PJMOA2Jpx9v-sGEyDuMu7ApODDBzbB4QQmx0M6Ryet6TJc_N05eru7fV0-FKun-8flYlVYTuRQ1EoRZ6Wy1KxbxoRgjvGaN4xbXitGFRFEjq9K1a6uhJC0EWZtrKpaB5Vo-ByJifu7NEGrt8n3Jn1rSvTBkN7oyZA-GNKTobF2M9Vg3Lb3kHS2HoIF5xPYQbvo_wf8AP67cdA</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>SLR: A million-scale comprehensive crossword dataset for simultaneous learning and reasoning</title><source>Elsevier</source><creator>Wang, Chao ; Zhu, Tinghui ; Li, Zhixu ; Liu, Jingping</creator><creatorcontrib>Wang, Chao ; Zhu, Tinghui ; Li, Zhixu ; Liu, Jingping</creatorcontrib><description>The progress of the natural language understanding (NLU) community has put forward higher demands for the knowledge reserve and reasoning ability of the model. However, existing schemes for improving model capabilities often split them into two separate tasks. The task of enabling models to learn and reason about knowledge simultaneously has not received sufficient attention. In this paper, we propose a novel crossword-based NLU task that imparts knowledge information to a model by solving crossword clues and simultaneously trains the model to infer new knowledge from existing knowledge. To this end, we construct a comprehensive crossword dataset SLR containing more than 4 million unique clue-answer pairs. Compared to existing crossword datasets, SLR is more comprehensive and contains linguistic knowledge, expertise in various fields, and commonsense knowledge. Meanwhile, to evaluate the reasoning ability of the model, we design clever details in the reasoning of the answers, most clues require the solver to reason through two or more pieces of knowledge to arrive at an answer. We analyze the composition of the dataset and the similarities and differences of the various types of clues via sampling and consider various data partitioning methods to enhance the generalization ability of the training set. Furthermore, we test the performance of several different advanced models and methods on this dataset and analyze the strengths and weaknesses of each. An interesting conclusion is that even powerful language models perform poorly on tasks that require reasoning.</description><identifier>ISSN: 0925-2312</identifier><identifier>EISSN: 1872-8286</identifier><identifier>DOI: 10.1016/j.neucom.2023.126591</identifier><language>eng</language><publisher>Elsevier B.V</publisher><subject>Crossword puzzle ; Knowledge reasoning ; Language model ; Open-domain question answering</subject><ispartof>Neurocomputing (Amsterdam), 2023-10, Vol.554, p.126591, Article 126591</ispartof><rights>2023 Elsevier B.V.</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c306t-7880dc68c1abf22442d2373923c3782180406abf587d75446194abac85fde5493</citedby><cites>FETCH-LOGICAL-c306t-7880dc68c1abf22442d2373923c3782180406abf587d75446194abac85fde5493</cites><orcidid>0000-0003-4843-1953</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>314,780,784,27924,27925</link.rule.ids></links><search><creatorcontrib>Wang, Chao</creatorcontrib><creatorcontrib>Zhu, Tinghui</creatorcontrib><creatorcontrib>Li, Zhixu</creatorcontrib><creatorcontrib>Liu, Jingping</creatorcontrib><title>SLR: A million-scale comprehensive crossword dataset for simultaneous learning and reasoning</title><title>Neurocomputing (Amsterdam)</title><description>The progress of the natural language understanding (NLU) community has put forward higher demands for the knowledge reserve and reasoning ability of the model. However, existing schemes for improving model capabilities often split them into two separate tasks. The task of enabling models to learn and reason about knowledge simultaneously has not received sufficient attention. In this paper, we propose a novel crossword-based NLU task that imparts knowledge information to a model by solving crossword clues and simultaneously trains the model to infer new knowledge from existing knowledge. To this end, we construct a comprehensive crossword dataset SLR containing more than 4 million unique clue-answer pairs. Compared to existing crossword datasets, SLR is more comprehensive and contains linguistic knowledge, expertise in various fields, and commonsense knowledge. Meanwhile, to evaluate the reasoning ability of the model, we design clever details in the reasoning of the answers, most clues require the solver to reason through two or more pieces of knowledge to arrive at an answer. We analyze the composition of the dataset and the similarities and differences of the various types of clues via sampling and consider various data partitioning methods to enhance the generalization ability of the training set. Furthermore, we test the performance of several different advanced models and methods on this dataset and analyze the strengths and weaknesses of each. An interesting conclusion is that even powerful language models perform poorly on tasks that require reasoning.</description><subject>Crossword puzzle</subject><subject>Knowledge reasoning</subject><subject>Language model</subject><subject>Open-domain question answering</subject><issn>0925-2312</issn><issn>1872-8286</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2023</creationdate><recordtype>article</recordtype><recordid>eNp9kNtKxDAURYMoOI7-gQ_5gdbcmqY-CMPgDQYEL29CyCSnmqFNhqQz4t_bsT77dNgc9mKzELqkpKSEyqtNGWBnY18ywnhJmawaeoRmVNWsUEzJYzQjDasKxik7RWc5bwihNWXNDL2_rJ6v8QL3vut8DEW2pgM8srYJPiFkvx9Tijl_xeSwM4PJMOA2Jpx9v-sGEyDuMu7ApODDBzbB4QQmx0M6Ryet6TJc_N05eru7fV0-FKun-8flYlVYTuRQ1EoRZ6Wy1KxbxoRgjvGaN4xbXitGFRFEjq9K1a6uhJC0EWZtrKpaB5Vo-ByJifu7NEGrt8n3Jn1rSvTBkN7oyZA-GNKTobF2M9Vg3Lb3kHS2HoIF5xPYQbvo_wf8AP67cdA</recordid><startdate>20231014</startdate><enddate>20231014</enddate><creator>Wang, Chao</creator><creator>Zhu, Tinghui</creator><creator>Li, Zhixu</creator><creator>Liu, Jingping</creator><general>Elsevier B.V</general><scope>AAYXX</scope><scope>CITATION</scope><orcidid>https://orcid.org/0000-0003-4843-1953</orcidid></search><sort><creationdate>20231014</creationdate><title>SLR: A million-scale comprehensive crossword dataset for simultaneous learning and reasoning</title><author>Wang, Chao ; Zhu, Tinghui ; Li, Zhixu ; Liu, Jingping</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c306t-7880dc68c1abf22442d2373923c3782180406abf587d75446194abac85fde5493</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2023</creationdate><topic>Crossword puzzle</topic><topic>Knowledge reasoning</topic><topic>Language model</topic><topic>Open-domain question answering</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Wang, Chao</creatorcontrib><creatorcontrib>Zhu, Tinghui</creatorcontrib><creatorcontrib>Li, Zhixu</creatorcontrib><creatorcontrib>Liu, Jingping</creatorcontrib><collection>CrossRef</collection><jtitle>Neurocomputing (Amsterdam)</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Wang, Chao</au><au>Zhu, Tinghui</au><au>Li, Zhixu</au><au>Liu, Jingping</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>SLR: A million-scale comprehensive crossword dataset for simultaneous learning and reasoning</atitle><jtitle>Neurocomputing (Amsterdam)</jtitle><date>2023-10-14</date><risdate>2023</risdate><volume>554</volume><spage>126591</spage><pages>126591-</pages><artnum>126591</artnum><issn>0925-2312</issn><eissn>1872-8286</eissn><abstract>The progress of the natural language understanding (NLU) community has put forward higher demands for the knowledge reserve and reasoning ability of the model. However, existing schemes for improving model capabilities often split them into two separate tasks. The task of enabling models to learn and reason about knowledge simultaneously has not received sufficient attention. In this paper, we propose a novel crossword-based NLU task that imparts knowledge information to a model by solving crossword clues and simultaneously trains the model to infer new knowledge from existing knowledge. To this end, we construct a comprehensive crossword dataset SLR containing more than 4 million unique clue-answer pairs. Compared to existing crossword datasets, SLR is more comprehensive and contains linguistic knowledge, expertise in various fields, and commonsense knowledge. Meanwhile, to evaluate the reasoning ability of the model, we design clever details in the reasoning of the answers, most clues require the solver to reason through two or more pieces of knowledge to arrive at an answer. We analyze the composition of the dataset and the similarities and differences of the various types of clues via sampling and consider various data partitioning methods to enhance the generalization ability of the training set. Furthermore, we test the performance of several different advanced models and methods on this dataset and analyze the strengths and weaknesses of each. An interesting conclusion is that even powerful language models perform poorly on tasks that require reasoning.</abstract><pub>Elsevier B.V</pub><doi>10.1016/j.neucom.2023.126591</doi><orcidid>https://orcid.org/0000-0003-4843-1953</orcidid></addata></record>
fulltext fulltext
identifier ISSN: 0925-2312
ispartof Neurocomputing (Amsterdam), 2023-10, Vol.554, p.126591, Article 126591
issn 0925-2312
1872-8286
language eng
recordid cdi_crossref_primary_10_1016_j_neucom_2023_126591
source Elsevier
subjects Crossword puzzle
Knowledge reasoning
Language model
Open-domain question answering
title SLR: A million-scale comprehensive crossword dataset for simultaneous learning and reasoning
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-24T18%3A37%3A09IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-elsevier_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=SLR:%20A%20million-scale%20comprehensive%20crossword%20dataset%20for%20simultaneous%20learning%20and%20reasoning&rft.jtitle=Neurocomputing%20(Amsterdam)&rft.au=Wang,%20Chao&rft.date=2023-10-14&rft.volume=554&rft.spage=126591&rft.pages=126591-&rft.artnum=126591&rft.issn=0925-2312&rft.eissn=1872-8286&rft_id=info:doi/10.1016/j.neucom.2023.126591&rft_dat=%3Celsevier_cross%3ES0925231223007142%3C/elsevier_cross%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-c306t-7880dc68c1abf22442d2373923c3782180406abf587d75446194abac85fde5493%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true