Multi-modes for Detecting Experimental Measurement Error

Experiments should be designed to facilitate the detection of experimental measurement error. To this end, we advocate the implementation of identical experimental protocols employing diverse experimental modes. We suggest iterative nonparametric estimation techniques for assessing the magnitude of...

Full description

Saved in:
Bibliographic Details
Published in:Political analysis 2020-04, Vol.28 (2), p.263-283
Main Authors: Duch, Raymond, Laroze, Denise, Robinson, Thomas, Beramendi, Pablo
Format: Article
Language:English
Subjects:
Earnings
Estimation
Experiments
Heterogeneity
Internet
Measurement errors
Researchers
Social networks
Social sciences
Statistical analysis
Workers
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by cdi_FETCH-LOGICAL-c299t-46e7582cb7eb087b5f5cef6eb3dc1eb241f5e38cfcb40ff0600f32cd14af28d43
cites cdi_FETCH-LOGICAL-c299t-46e7582cb7eb087b5f5cef6eb3dc1eb241f5e38cfcb40ff0600f32cd14af28d43
container_end_page 283
container_issue 2
container_start_page 263
container_title Political analysis
container_volume 28
creator Duch, Raymond
Laroze, Denise
Robinson, Thomas
Beramendi, Pablo
description Experiments should be designed to facilitate the detection of experimental measurement error. To this end, we advocate the implementation of identical experimental protocols employing diverse experimental modes. We suggest iterative nonparametric estimation techniques for assessing the magnitude of heterogeneous treatment effects across these modes. And we propose two diagnostic strategies—measurement metrics embedded in experiments, and measurement experiments—that help assess whether any observed heterogeneity reflects experimental measurement error. To illustrate our argument, first we conduct and analyze results from four identical interactive experiments: in the lab; online with subjects from the CESS lab subject pool; online with an online subject pool; and online with MTurk workers. Second, we implement a measurement experiment in India with CESS Online subjects and MTurk workers.
doi_str_mv 10.1017/pan.2019.34
format article
fullrecord <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_journals_2369744085</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><cupid>10_1017_pan_2019_34</cupid><sourcerecordid>2369744085</sourcerecordid><originalsourceid>FETCH-LOGICAL-c299t-46e7582cb7eb087b5f5cef6eb3dc1eb241f5e38cfcb40ff0600f32cd14af28d43</originalsourceid><addsrcrecordid>eNptkE1LxDAQhoMouK6e_AMFj5KarzbJUdb1A3bxoueQpJOlS7etSQv6723ZBS-e5h14eGd4ELqlJKeEyofetjkjVOdcnKEFFbLEQit9PmUiJKZayUt0ldKeTLTUeoHUdmyGGh-6ClIWupg9wQB-qNtdtv7uIdYHaAfbZFuwaYwwb9k6xi5eo4tgmwQ3p7lEn8_rj9Ur3ry_vK0eN9gzrQcsSpCFYt5JcERJV4TCQyjB8cpTcEzQUABXPngnSAikJCRw5isqbGCqEnyJ7o69fey-RkiD2XdjbKeThvFSSyGIKibq_kj52KUUIZh-et3GH0OJmdWYSY2Z1Rg-d-ITbQ8u1tUO_kr_438BbEhmOw</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2369744085</pqid></control><display><type>article</type><title>Multi-modes for Detecting Experimental Measurement Error</title><source>JSTOR Archival Journals and Primary Sources Collection【Remote access available】</source><source>Social Science Premium Collection</source><source>Politics Collection</source><source>Worldwide Political Science Abstracts</source><source>Cambridge University Press</source><creator>Duch, Raymond ; Laroze, Denise ; Robinson, Thomas ; Beramendi, Pablo</creator><creatorcontrib>Duch, Raymond ; Laroze, Denise ; Robinson, Thomas ; Beramendi, Pablo</creatorcontrib><description>Experiments should be designed to facilitate the detection of experimental measurement error. To this end, we advocate the implementation of identical experimental protocols employing diverse experimental modes. We suggest iterative nonparametric estimation techniques for assessing the magnitude of heterogeneous treatment effects across these modes. And we propose two diagnostic strategies—measurement metrics embedded in experiments, and measurement experiments—that help assess whether any observed heterogeneity reflects experimental measurement error. To illustrate our argument, first we conduct and analyze results from four identical interactive experiments: in the lab; online with subjects from the CESS lab subject pool; online with an online subject pool; and online with MTurk workers. Second, we implement a measurement experiment in India with CESS Online subjects and MTurk workers.</description><identifier>ISSN: 1047-1987</identifier><identifier>EISSN: 1476-4989</identifier><identifier>DOI: 10.1017/pan.2019.34</identifier><language>eng</language><publisher>New York, USA: Cambridge University Press</publisher><subject>Earnings ; Estimation ; Experiments ; Heterogeneity ; Internet ; Measurement errors ; Researchers ; Social networks ; Social sciences ; Statistical analysis ; Workers</subject><ispartof>Political analysis, 2020-04, Vol.28 (2), p.263-283</ispartof><rights>Copyright © The Author(s) 2019. Published by Cambridge University Press on behalf of the Society for Political Methodology.</rights><rights>Copyright Cambridge University Press Apr 2020</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c299t-46e7582cb7eb087b5f5cef6eb3dc1eb241f5e38cfcb40ff0600f32cd14af28d43</citedby><cites>FETCH-LOGICAL-c299t-46e7582cb7eb087b5f5cef6eb3dc1eb241f5e38cfcb40ff0600f32cd14af28d43</cites><orcidid>0000-0002-6138-2570 ; 0000-0001-7097-1599 ; 0000-0002-1166-7674</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://www.proquest.com/docview/2369744085/fulltextPDF?pq-origsite=primo$$EPDF$$P50$$Gproquest$$H</linktopdf><linktohtml>$$Uhttps://www.proquest.com/docview/2369744085?pq-origsite=primo$$EHTML$$P50$$Gproquest$$H</linktohtml><link.rule.ids>314,780,784,12845,21387,21394,27924,27925,33611,33985,43733,43948,72960,74221,74468</link.rule.ids></links><search><creatorcontrib>Duch, Raymond</creatorcontrib><creatorcontrib>Laroze, Denise</creatorcontrib><creatorcontrib>Robinson, Thomas</creatorcontrib><creatorcontrib>Beramendi, Pablo</creatorcontrib><title>Multi-modes for Detecting Experimental Measurement Error</title><title>Political analysis</title><addtitle>Polit. Anal</addtitle><description>Experiments should be designed to facilitate the detection of experimental measurement error. To this end, we advocate the implementation of identical experimental protocols employing diverse experimental modes. We suggest iterative nonparametric estimation techniques for assessing the magnitude of heterogeneous treatment effects across these modes. And we propose two diagnostic strategies—measurement metrics embedded in experiments, and measurement experiments—that help assess whether any observed heterogeneity reflects experimental measurement error. To illustrate our argument, first we conduct and analyze results from four identical interactive experiments: in the lab; online with subjects from the CESS lab subject pool; online with an online subject pool; and online with MTurk workers. Second, we implement a measurement experiment in India with CESS Online subjects and MTurk workers.</description><subject>Earnings</subject><subject>Estimation</subject><subject>Experiments</subject><subject>Heterogeneity</subject><subject>Internet</subject><subject>Measurement errors</subject><subject>Researchers</subject><subject>Social networks</subject><subject>Social sciences</subject><subject>Statistical analysis</subject><subject>Workers</subject><issn>1047-1987</issn><issn>1476-4989</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2020</creationdate><recordtype>article</recordtype><sourceid>7UB</sourceid><sourceid>ALSLI</sourceid><sourceid>DPSOV</sourceid><sourceid>M2L</sourceid><sourceid>M2R</sourceid><recordid>eNptkE1LxDAQhoMouK6e_AMFj5KarzbJUdb1A3bxoueQpJOlS7etSQv6723ZBS-e5h14eGd4ELqlJKeEyofetjkjVOdcnKEFFbLEQit9PmUiJKZayUt0ldKeTLTUeoHUdmyGGh-6ClIWupg9wQB-qNtdtv7uIdYHaAfbZFuwaYwwb9k6xi5eo4tgmwQ3p7lEn8_rj9Ur3ry_vK0eN9gzrQcsSpCFYt5JcERJV4TCQyjB8cpTcEzQUABXPngnSAikJCRw5isqbGCqEnyJ7o69fey-RkiD2XdjbKeThvFSSyGIKibq_kj52KUUIZh-et3GH0OJmdWYSY2Z1Rg-d-ITbQ8u1tUO_kr_438BbEhmOw</recordid><startdate>202004</startdate><enddate>202004</enddate><creator>Duch, Raymond</creator><creator>Laroze, Denise</creator><creator>Robinson, Thomas</creator><creator>Beramendi, Pablo</creator><general>Cambridge University Press</general><scope>AAYXX</scope><scope>CITATION</scope><scope>0-V</scope><scope>3V.</scope><scope>7UB</scope><scope>7XB</scope><scope>88F</scope><scope>88J</scope><scope>8FK</scope><scope>8G5</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>ALSLI</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>CCPQU</scope><scope>DPSOV</scope><scope>DWQXO</scope><scope>GNUQQ</scope><scope>GUQSH</scope><scope>KC-</scope><scope>M1Q</scope><scope>M2L</scope><scope>M2O</scope><scope>M2R</scope><scope>MBDVC</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>Q9U</scope><orcidid>https://orcid.org/0000-0002-6138-2570</orcidid><orcidid>https://orcid.org/0000-0001-7097-1599</orcidid><orcidid>https://orcid.org/0000-0002-1166-7674</orcidid></search><sort><creationdate>202004</creationdate><title>Multi-modes for Detecting Experimental Measurement Error</title><author>Duch, Raymond ; Laroze, Denise ; Robinson, Thomas ; Beramendi, Pablo</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c299t-46e7582cb7eb087b5f5cef6eb3dc1eb241f5e38cfcb40ff0600f32cd14af28d43</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2020</creationdate><topic>Earnings</topic><topic>Estimation</topic><topic>Experiments</topic><topic>Heterogeneity</topic><topic>Internet</topic><topic>Measurement errors</topic><topic>Researchers</topic><topic>Social networks</topic><topic>Social sciences</topic><topic>Statistical analysis</topic><topic>Workers</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Duch, Raymond</creatorcontrib><creatorcontrib>Laroze, Denise</creatorcontrib><creatorcontrib>Robinson, Thomas</creatorcontrib><creatorcontrib>Beramendi, Pablo</creatorcontrib><collection>CrossRef</collection><collection>ProQuest Social Sciences Premium Collection</collection><collection>ProQuest Central (Corporate)</collection><collection>Worldwide Political Science Abstracts</collection><collection>ProQuest Central (purchase pre-March 2016)</collection><collection>Military Database (Alumni Edition)</collection><collection>Social Science Database (Alumni Edition)</collection><collection>ProQuest Central (Alumni) (purchase pre-March 2016)</collection><collection>Research Library (Alumni Edition)</collection><collection>ProQuest Central (Alumni)</collection><collection>ProQuest Central</collection><collection>Social Science Premium Collection</collection><collection>ProQuest Central Essentials</collection><collection>AUTh Library subscriptions: ProQuest Central</collection><collection>ProQuest One Community College</collection><collection>Politics Collection</collection><collection>ProQuest Central Korea</collection><collection>ProQuest Central Student</collection><collection>Research Library Prep</collection><collection>ProQuest Politics Collection</collection><collection>ProQuest Military Database</collection><collection>Political Science Database</collection><collection>ProQuest_Research Library</collection><collection>Social Science Database</collection><collection>Research Library (Corporate)</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>ProQuest Central Basic</collection><jtitle>Political analysis</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Duch, Raymond</au><au>Laroze, Denise</au><au>Robinson, Thomas</au><au>Beramendi, Pablo</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Multi-modes for Detecting Experimental Measurement Error</atitle><jtitle>Political analysis</jtitle><addtitle>Polit. Anal</addtitle><date>2020-04</date><risdate>2020</risdate><volume>28</volume><issue>2</issue><spage>263</spage><epage>283</epage><pages>263-283</pages><issn>1047-1987</issn><eissn>1476-4989</eissn><abstract>Experiments should be designed to facilitate the detection of experimental measurement error. To this end, we advocate the implementation of identical experimental protocols employing diverse experimental modes. We suggest iterative nonparametric estimation techniques for assessing the magnitude of heterogeneous treatment effects across these modes. And we propose two diagnostic strategies—measurement metrics embedded in experiments, and measurement experiments—that help assess whether any observed heterogeneity reflects experimental measurement error. To illustrate our argument, first we conduct and analyze results from four identical interactive experiments: in the lab; online with subjects from the CESS lab subject pool; online with an online subject pool; and online with MTurk workers. Second, we implement a measurement experiment in India with CESS Online subjects and MTurk workers.</abstract><cop>New York, USA</cop><pub>Cambridge University Press</pub><doi>10.1017/pan.2019.34</doi><tpages>21</tpages><orcidid>https://orcid.org/0000-0002-6138-2570</orcidid><orcidid>https://orcid.org/0000-0001-7097-1599</orcidid><orcidid>https://orcid.org/0000-0002-1166-7674</orcidid></addata></record>
fulltext fulltext
identifier ISSN: 1047-1987
ispartof Political analysis, 2020-04, Vol.28 (2), p.263-283
issn 1047-1987
1476-4989
language eng
recordid cdi_proquest_journals_2369744085
source JSTOR Archival Journals and Primary Sources Collection【Remote access available】; Social Science Premium Collection; Politics Collection; Worldwide Political Science Abstracts; Cambridge University Press
subjects Earnings
Estimation
Experiments
Heterogeneity
Internet
Measurement errors
Researchers
Social networks
Social sciences
Statistical analysis
Workers
title Multi-modes for Detecting Experimental Measurement Error
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-27T02%3A25%3A10IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Multi-modes%20for%20Detecting%20Experimental%20Measurement%20Error&rft.jtitle=Political%20analysis&rft.au=Duch,%20Raymond&rft.date=2020-04&rft.volume=28&rft.issue=2&rft.spage=263&rft.epage=283&rft.pages=263-283&rft.issn=1047-1987&rft.eissn=1476-4989&rft_id=info:doi/10.1017/pan.2019.34&rft_dat=%3Cproquest_cross%3E2369744085%3C/proquest_cross%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-c299t-46e7582cb7eb087b5f5cef6eb3dc1eb241f5e38cfcb40ff0600f32cd14af28d43%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_pqid=2369744085&rft_id=info:pmid/&rft_cupid=10_1017_pan_2019_34&rfr_iscdi=true