Loading…
Data Contamination Report from the 2024 CONDA Shared Task
The 1st Workshop on Data Contamination (CONDA 2024) focuses on all relevant aspects of data contamination in natural language processing, where data contamination is understood as situations where evaluation data is included in pre-training corpora used to train large scale models, compromising eval...
Saved in:
Published in: | arXiv.org 2024-08 |
---|---|
Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , |
Format: | Article |
Language: | English |
Subjects: | |
Online Access: | Get full text |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
cited_by | |
---|---|
cites | |
container_end_page | |
container_issue | |
container_start_page | |
container_title | arXiv.org |
container_volume | |
creator | Sainz, Oscar García-Ferrero, Iker Jacovi, Alon Jon Ander Campos Yanai Elazar Agirre, Eneko Goldberg, Yoav Wei-Lin, Chen Chim, Jenny Leshem Choshen D'Amico-Wong, Luca Dell, Melissa Fan, Run-Ze Golchin, Shahriar Li, Yucheng Liu, Pengfei Pahwa, Bhavish Prabhu, Ameya Sharma, Suryansh Silcock, Emily Solonko, Kateryna Stap, David Surdeanu, Mihai Yu-Min, Tseng Udandarao, Vishaal Wang, Zengzhi Xu, Ruijie Yang, Jinglin |
description | The 1st Workshop on Data Contamination (CONDA 2024) focuses on all relevant aspects of data contamination in natural language processing, where data contamination is understood as situations where evaluation data is included in pre-training corpora used to train large scale models, compromising evaluation results. The workshop fostered a shared task to collect evidence on data contamination in current available datasets and models. The goal of the shared task and associated database is to assist the community in understanding the extent of the problem and to assist researchers in avoiding reporting evaluation results on known contaminated resources. The shared task provides a structured, centralized public database for the collection of contamination evidence, open to contributions from the community via GitHub pool requests. This first compilation paper is based on 566 reported entries over 91 contaminated sources from a total of 23 contributors. The details of the individual contamination events are available in the platform. The platform continues to be online, open to contributions from the community. |
format | article |
fullrecord | <record><control><sourceid>proquest</sourceid><recordid>TN_cdi_proquest_journals_3087032837</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>3087032837</sourcerecordid><originalsourceid>FETCH-proquest_journals_30870328373</originalsourceid><addsrcrecordid>eNqNysEKgkAQgOElCJLyHQY6C9uMtnYMLToVlHcZcEUtd213ff869ACd_sP3L0SERLskTxFXIvZ-kFLiXmGWUSQOJQeGwprAY2849NbAXU_WBWidHSF0GlBiCsXtWh7h0bHTDVTsnxuxbPnldfzrWmzPp6q4JJOz71n7UA92duZLNclcScKcFP13fQChwjRm</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>3087032837</pqid></control><display><type>article</type><title>Data Contamination Report from the 2024 CONDA Shared Task</title><source>Publicly Available Content Database</source><creator>Sainz, Oscar ; García-Ferrero, Iker ; Jacovi, Alon ; Jon Ander Campos ; Yanai Elazar ; Agirre, Eneko ; Goldberg, Yoav ; Wei-Lin, Chen ; Chim, Jenny ; Leshem Choshen ; D'Amico-Wong, Luca ; Dell, Melissa ; Fan, Run-Ze ; Golchin, Shahriar ; Li, Yucheng ; Liu, Pengfei ; Pahwa, Bhavish ; Prabhu, Ameya ; Sharma, Suryansh ; Silcock, Emily ; Solonko, Kateryna ; Stap, David ; Surdeanu, Mihai ; Yu-Min, Tseng ; Udandarao, Vishaal ; Wang, Zengzhi ; Xu, Ruijie ; Yang, Jinglin</creator><creatorcontrib>Sainz, Oscar ; García-Ferrero, Iker ; Jacovi, Alon ; Jon Ander Campos ; Yanai Elazar ; Agirre, Eneko ; Goldberg, Yoav ; Wei-Lin, Chen ; Chim, Jenny ; Leshem Choshen ; D'Amico-Wong, Luca ; Dell, Melissa ; Fan, Run-Ze ; Golchin, Shahriar ; Li, Yucheng ; Liu, Pengfei ; Pahwa, Bhavish ; Prabhu, Ameya ; Sharma, Suryansh ; Silcock, Emily ; Solonko, Kateryna ; Stap, David ; Surdeanu, Mihai ; Yu-Min, Tseng ; Udandarao, Vishaal ; Wang, Zengzhi ; Xu, Ruijie ; Yang, Jinglin</creatorcontrib><description>The 1st Workshop on Data Contamination (CONDA 2024) focuses on all relevant aspects of data contamination in natural language processing, where data contamination is understood as situations where evaluation data is included in pre-training corpora used to train large scale models, compromising evaluation results. The workshop fostered a shared task to collect evidence on data contamination in current available datasets and models. The goal of the shared task and associated database is to assist the community in understanding the extent of the problem and to assist researchers in avoiding reporting evaluation results on known contaminated resources. The shared task provides a structured, centralized public database for the collection of contamination evidence, open to contributions from the community via GitHub pool requests. This first compilation paper is based on 566 reported entries over 91 contaminated sources from a total of 23 contributors. The details of the individual contamination events are available in the platform. The platform continues to be online, open to contributions from the community.</description><identifier>EISSN: 2331-8422</identifier><language>eng</language><publisher>Ithaca: Cornell University Library, arXiv.org</publisher><subject>Contamination ; Natural language processing ; Scale models ; Workshops</subject><ispartof>arXiv.org, 2024-08</ispartof><rights>2024. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://www.proquest.com/docview/3087032837?pq-origsite=primo$$EHTML$$P50$$Gproquest$$Hfree_for_read</linktohtml><link.rule.ids>780,784,25753,37012,44590</link.rule.ids></links><search><creatorcontrib>Sainz, Oscar</creatorcontrib><creatorcontrib>García-Ferrero, Iker</creatorcontrib><creatorcontrib>Jacovi, Alon</creatorcontrib><creatorcontrib>Jon Ander Campos</creatorcontrib><creatorcontrib>Yanai Elazar</creatorcontrib><creatorcontrib>Agirre, Eneko</creatorcontrib><creatorcontrib>Goldberg, Yoav</creatorcontrib><creatorcontrib>Wei-Lin, Chen</creatorcontrib><creatorcontrib>Chim, Jenny</creatorcontrib><creatorcontrib>Leshem Choshen</creatorcontrib><creatorcontrib>D'Amico-Wong, Luca</creatorcontrib><creatorcontrib>Dell, Melissa</creatorcontrib><creatorcontrib>Fan, Run-Ze</creatorcontrib><creatorcontrib>Golchin, Shahriar</creatorcontrib><creatorcontrib>Li, Yucheng</creatorcontrib><creatorcontrib>Liu, Pengfei</creatorcontrib><creatorcontrib>Pahwa, Bhavish</creatorcontrib><creatorcontrib>Prabhu, Ameya</creatorcontrib><creatorcontrib>Sharma, Suryansh</creatorcontrib><creatorcontrib>Silcock, Emily</creatorcontrib><creatorcontrib>Solonko, Kateryna</creatorcontrib><creatorcontrib>Stap, David</creatorcontrib><creatorcontrib>Surdeanu, Mihai</creatorcontrib><creatorcontrib>Yu-Min, Tseng</creatorcontrib><creatorcontrib>Udandarao, Vishaal</creatorcontrib><creatorcontrib>Wang, Zengzhi</creatorcontrib><creatorcontrib>Xu, Ruijie</creatorcontrib><creatorcontrib>Yang, Jinglin</creatorcontrib><title>Data Contamination Report from the 2024 CONDA Shared Task</title><title>arXiv.org</title><description>The 1st Workshop on Data Contamination (CONDA 2024) focuses on all relevant aspects of data contamination in natural language processing, where data contamination is understood as situations where evaluation data is included in pre-training corpora used to train large scale models, compromising evaluation results. The workshop fostered a shared task to collect evidence on data contamination in current available datasets and models. The goal of the shared task and associated database is to assist the community in understanding the extent of the problem and to assist researchers in avoiding reporting evaluation results on known contaminated resources. The shared task provides a structured, centralized public database for the collection of contamination evidence, open to contributions from the community via GitHub pool requests. This first compilation paper is based on 566 reported entries over 91 contaminated sources from a total of 23 contributors. The details of the individual contamination events are available in the platform. The platform continues to be online, open to contributions from the community.</description><subject>Contamination</subject><subject>Natural language processing</subject><subject>Scale models</subject><subject>Workshops</subject><issn>2331-8422</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><sourceid>PIMPY</sourceid><recordid>eNqNysEKgkAQgOElCJLyHQY6C9uMtnYMLToVlHcZcEUtd213ff869ACd_sP3L0SERLskTxFXIvZ-kFLiXmGWUSQOJQeGwprAY2849NbAXU_WBWidHSF0GlBiCsXtWh7h0bHTDVTsnxuxbPnldfzrWmzPp6q4JJOz71n7UA92duZLNclcScKcFP13fQChwjRm</recordid><startdate>20240804</startdate><enddate>20240804</enddate><creator>Sainz, Oscar</creator><creator>García-Ferrero, Iker</creator><creator>Jacovi, Alon</creator><creator>Jon Ander Campos</creator><creator>Yanai Elazar</creator><creator>Agirre, Eneko</creator><creator>Goldberg, Yoav</creator><creator>Wei-Lin, Chen</creator><creator>Chim, Jenny</creator><creator>Leshem Choshen</creator><creator>D'Amico-Wong, Luca</creator><creator>Dell, Melissa</creator><creator>Fan, Run-Ze</creator><creator>Golchin, Shahriar</creator><creator>Li, Yucheng</creator><creator>Liu, Pengfei</creator><creator>Pahwa, Bhavish</creator><creator>Prabhu, Ameya</creator><creator>Sharma, Suryansh</creator><creator>Silcock, Emily</creator><creator>Solonko, Kateryna</creator><creator>Stap, David</creator><creator>Surdeanu, Mihai</creator><creator>Yu-Min, Tseng</creator><creator>Udandarao, Vishaal</creator><creator>Wang, Zengzhi</creator><creator>Xu, Ruijie</creator><creator>Yang, Jinglin</creator><general>Cornell University Library, arXiv.org</general><scope>8FE</scope><scope>8FG</scope><scope>ABJCF</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BGLVJ</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>HCIFZ</scope><scope>L6V</scope><scope>M7S</scope><scope>PIMPY</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>PTHSS</scope></search><sort><creationdate>20240804</creationdate><title>Data Contamination Report from the 2024 CONDA Shared Task</title><author>Sainz, Oscar ; García-Ferrero, Iker ; Jacovi, Alon ; Jon Ander Campos ; Yanai Elazar ; Agirre, Eneko ; Goldberg, Yoav ; Wei-Lin, Chen ; Chim, Jenny ; Leshem Choshen ; D'Amico-Wong, Luca ; Dell, Melissa ; Fan, Run-Ze ; Golchin, Shahriar ; Li, Yucheng ; Liu, Pengfei ; Pahwa, Bhavish ; Prabhu, Ameya ; Sharma, Suryansh ; Silcock, Emily ; Solonko, Kateryna ; Stap, David ; Surdeanu, Mihai ; Yu-Min, Tseng ; Udandarao, Vishaal ; Wang, Zengzhi ; Xu, Ruijie ; Yang, Jinglin</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-proquest_journals_30870328373</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><topic>Contamination</topic><topic>Natural language processing</topic><topic>Scale models</topic><topic>Workshops</topic><toplevel>online_resources</toplevel><creatorcontrib>Sainz, Oscar</creatorcontrib><creatorcontrib>García-Ferrero, Iker</creatorcontrib><creatorcontrib>Jacovi, Alon</creatorcontrib><creatorcontrib>Jon Ander Campos</creatorcontrib><creatorcontrib>Yanai Elazar</creatorcontrib><creatorcontrib>Agirre, Eneko</creatorcontrib><creatorcontrib>Goldberg, Yoav</creatorcontrib><creatorcontrib>Wei-Lin, Chen</creatorcontrib><creatorcontrib>Chim, Jenny</creatorcontrib><creatorcontrib>Leshem Choshen</creatorcontrib><creatorcontrib>D'Amico-Wong, Luca</creatorcontrib><creatorcontrib>Dell, Melissa</creatorcontrib><creatorcontrib>Fan, Run-Ze</creatorcontrib><creatorcontrib>Golchin, Shahriar</creatorcontrib><creatorcontrib>Li, Yucheng</creatorcontrib><creatorcontrib>Liu, Pengfei</creatorcontrib><creatorcontrib>Pahwa, Bhavish</creatorcontrib><creatorcontrib>Prabhu, Ameya</creatorcontrib><creatorcontrib>Sharma, Suryansh</creatorcontrib><creatorcontrib>Silcock, Emily</creatorcontrib><creatorcontrib>Solonko, Kateryna</creatorcontrib><creatorcontrib>Stap, David</creatorcontrib><creatorcontrib>Surdeanu, Mihai</creatorcontrib><creatorcontrib>Yu-Min, Tseng</creatorcontrib><creatorcontrib>Udandarao, Vishaal</creatorcontrib><creatorcontrib>Wang, Zengzhi</creatorcontrib><creatorcontrib>Xu, Ruijie</creatorcontrib><creatorcontrib>Yang, Jinglin</creatorcontrib><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>Materials Science & Engineering Collection</collection><collection>ProQuest Central (Alumni)</collection><collection>ProQuest Central</collection><collection>ProQuest Central Essentials</collection><collection>AUTh Library subscriptions: ProQuest Central</collection><collection>Technology Collection</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central Korea</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Engineering Collection</collection><collection>Engineering Database</collection><collection>Publicly Available Content Database</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>Engineering collection</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Sainz, Oscar</au><au>García-Ferrero, Iker</au><au>Jacovi, Alon</au><au>Jon Ander Campos</au><au>Yanai Elazar</au><au>Agirre, Eneko</au><au>Goldberg, Yoav</au><au>Wei-Lin, Chen</au><au>Chim, Jenny</au><au>Leshem Choshen</au><au>D'Amico-Wong, Luca</au><au>Dell, Melissa</au><au>Fan, Run-Ze</au><au>Golchin, Shahriar</au><au>Li, Yucheng</au><au>Liu, Pengfei</au><au>Pahwa, Bhavish</au><au>Prabhu, Ameya</au><au>Sharma, Suryansh</au><au>Silcock, Emily</au><au>Solonko, Kateryna</au><au>Stap, David</au><au>Surdeanu, Mihai</au><au>Yu-Min, Tseng</au><au>Udandarao, Vishaal</au><au>Wang, Zengzhi</au><au>Xu, Ruijie</au><au>Yang, Jinglin</au><format>book</format><genre>document</genre><ristype>GEN</ristype><atitle>Data Contamination Report from the 2024 CONDA Shared Task</atitle><jtitle>arXiv.org</jtitle><date>2024-08-04</date><risdate>2024</risdate><eissn>2331-8422</eissn><abstract>The 1st Workshop on Data Contamination (CONDA 2024) focuses on all relevant aspects of data contamination in natural language processing, where data contamination is understood as situations where evaluation data is included in pre-training corpora used to train large scale models, compromising evaluation results. The workshop fostered a shared task to collect evidence on data contamination in current available datasets and models. The goal of the shared task and associated database is to assist the community in understanding the extent of the problem and to assist researchers in avoiding reporting evaluation results on known contaminated resources. The shared task provides a structured, centralized public database for the collection of contamination evidence, open to contributions from the community via GitHub pool requests. This first compilation paper is based on 566 reported entries over 91 contaminated sources from a total of 23 contributors. The details of the individual contamination events are available in the platform. The platform continues to be online, open to contributions from the community.</abstract><cop>Ithaca</cop><pub>Cornell University Library, arXiv.org</pub><oa>free_for_read</oa></addata></record> |
fulltext | fulltext |
identifier | EISSN: 2331-8422 |
ispartof | arXiv.org, 2024-08 |
issn | 2331-8422 |
language | eng |
recordid | cdi_proquest_journals_3087032837 |
source | Publicly Available Content Database |
subjects | Contamination Natural language processing Scale models Workshops |
title | Data Contamination Report from the 2024 CONDA Shared Task |
url | http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-25T20%3A35%3A16IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest&rft_val_fmt=info:ofi/fmt:kev:mtx:book&rft.genre=document&rft.atitle=Data%20Contamination%20Report%20from%20the%202024%20CONDA%20Shared%20Task&rft.jtitle=arXiv.org&rft.au=Sainz,%20Oscar&rft.date=2024-08-04&rft.eissn=2331-8422&rft_id=info:doi/&rft_dat=%3Cproquest%3E3087032837%3C/proquest%3E%3Cgrp_id%3Ecdi_FETCH-proquest_journals_30870328373%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_pqid=3087032837&rft_id=info:pmid/&rfr_iscdi=true |