History Assessments of Thinking: A Validity Study

This article reports a validity study of History Assessments of Thinking (HATs), which are short, constructed-response assessments of historical thinking. In particular, this study focuses on aspects of cognitive validity, which is an examination of whether assessments tap the intended constructs. T...

Full description

Saved in:
Bibliographic Details
Published in:Cognition and instruction 2019-01, Vol.37 (1), p.118-144
Main Authors: Smith, Mark, Breakstone, Joel, Wineburg, Sam
Format: Article
Language:English
Subjects:
assessment
Cognitive Processes
Educational evaluation
Evaluation Methods
High School Students
Historical thinking
History
History education
History Instruction
Item Analysis
Multiple Choice Tests
Protocol Analysis
Student Evaluation
Test Items
Thinking Skills
Validity
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by cdi_FETCH-LOGICAL-c426t-4e7686bd3ec7389159d78b5db0408fcd472afcf0a33b2cac6ed8d6dbb9f5d8473
cites cdi_FETCH-LOGICAL-c426t-4e7686bd3ec7389159d78b5db0408fcd472afcf0a33b2cac6ed8d6dbb9f5d8473
container_end_page 144
container_issue 1
container_start_page 118
container_title Cognition and instruction
container_volume 37
creator Smith, Mark
Breakstone, Joel
Wineburg, Sam
description This article reports a validity study of History Assessments of Thinking (HATs), which are short, constructed-response assessments of historical thinking. In particular, this study focuses on aspects of cognitive validity, which is an examination of whether assessments tap the intended constructs. Think-aloud interviews with 26 high school students were used to examine the thinking elicited by 8 HATs and multiple-choice versions of these tasks. Results showed that although both HATs and multiple-choice items tapped historical thinking processes, HATs better reflected student proficiency in historical thinking than their multiple-choice counterparts. Item format also influenced the thinking elicited, with multiple-choice items eliciting more instances of construct-irrelevant reasoning than the constructed-response versions. Implications for history assessment are discussed.
doi_str_mv 10.1080/07370008.2018.1499646
format article
fullrecord <record><control><sourceid>proquest_eric_</sourceid><recordid>TN_cdi_eric_primary_EJ1208026</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ericid>EJ1208026</ericid><sourcerecordid>2188842104</sourcerecordid><originalsourceid>FETCH-LOGICAL-c426t-4e7686bd3ec7389159d78b5db0408fcd472afcf0a33b2cac6ed8d6dbb9f5d8473</originalsourceid><addsrcrecordid>eNp9kF1LwzAUhoMoOKc_YVDwujNfTVKvHEOdMvDCKd6FNB-auTUz6ZD-e1s6vfTqHHif9xx4AJggOEVQwCvICYcQiimGSEwRLUtG2REYoYLgnJXw7RiMeibvoVNwltK623CB-AighU9NiG02S8mmtLV1k7LgstWHrz99_X6dzbJXtfHGN2323OxNew5OnNoke3GYY_Byd7uaL_Ll0_3DfLbMNcWsyanlTLDKEKs5ESUqSsNFVZgKUiicNpRj5bSDipAKa6WZNcIwU1WlK4ygnIzB5XB3F8PX3qZGrsM-1t1LiZEQgmIEaUcVA6VjSClaJ3fRb1VsJYKytyN_7cjejjzY6XqToWej13-d20eEuw7u85sh97ULcau-Q9wY2ah2E6KLqtY-SfL_ix_nWXRW</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2188842104</pqid></control><display><type>article</type><title>History Assessments of Thinking: A Validity Study</title><source>Taylor &amp; Francis</source><source>ERIC</source><creator>Smith, Mark ; Breakstone, Joel ; Wineburg, Sam</creator><creatorcontrib>Smith, Mark ; Breakstone, Joel ; Wineburg, Sam</creatorcontrib><description>This article reports a validity study of History Assessments of Thinking (HATs), which are short, constructed-response assessments of historical thinking. In particular, this study focuses on aspects of cognitive validity, which is an examination of whether assessments tap the intended constructs. Think-aloud interviews with 26 high school students were used to examine the thinking elicited by 8 HATs and multiple-choice versions of these tasks. Results showed that although both HATs and multiple-choice items tapped historical thinking processes, HATs better reflected student proficiency in historical thinking than their multiple-choice counterparts. Item format also influenced the thinking elicited, with multiple-choice items eliciting more instances of construct-irrelevant reasoning than the constructed-response versions. Implications for history assessment are discussed.</description><identifier>ISSN: 0737-0008</identifier><identifier>EISSN: 1532-690X</identifier><identifier>DOI: 10.1080/07370008.2018.1499646</identifier><language>eng</language><publisher>Philadelphia: Routledge</publisher><subject>assessment ; Cognitive Processes ; Educational evaluation ; Evaluation Methods ; High School Students ; Historical thinking ; History ; History education ; History Instruction ; Item Analysis ; Multiple Choice Tests ; Protocol Analysis ; Student Evaluation ; Test Items ; Thinking Skills ; Validity</subject><ispartof>Cognition and instruction, 2019-01, Vol.37 (1), p.118-144</ispartof><rights>2018 Taylor &amp; Francis Group, LLC 2018</rights><rights>2018 Taylor &amp; Francis Group, LLC</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c426t-4e7686bd3ec7389159d78b5db0408fcd472afcf0a33b2cac6ed8d6dbb9f5d8473</citedby><cites>FETCH-LOGICAL-c426t-4e7686bd3ec7389159d78b5db0408fcd472afcf0a33b2cac6ed8d6dbb9f5d8473</cites><orcidid>0000-0002-1048-7786 ; 0000-0003-0468-6399 ; 0000-0002-4838-1522</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>314,780,784,27924,27925</link.rule.ids><backlink>$$Uhttp://eric.ed.gov/ERICWebPortal/detail?accno=EJ1208026$$DView record in ERIC$$Hfree_for_read</backlink></links><search><creatorcontrib>Smith, Mark</creatorcontrib><creatorcontrib>Breakstone, Joel</creatorcontrib><creatorcontrib>Wineburg, Sam</creatorcontrib><title>History Assessments of Thinking: A Validity Study</title><title>Cognition and instruction</title><description>This article reports a validity study of History Assessments of Thinking (HATs), which are short, constructed-response assessments of historical thinking. In particular, this study focuses on aspects of cognitive validity, which is an examination of whether assessments tap the intended constructs. Think-aloud interviews with 26 high school students were used to examine the thinking elicited by 8 HATs and multiple-choice versions of these tasks. Results showed that although both HATs and multiple-choice items tapped historical thinking processes, HATs better reflected student proficiency in historical thinking than their multiple-choice counterparts. Item format also influenced the thinking elicited, with multiple-choice items eliciting more instances of construct-irrelevant reasoning than the constructed-response versions. Implications for history assessment are discussed.</description><subject>assessment</subject><subject>Cognitive Processes</subject><subject>Educational evaluation</subject><subject>Evaluation Methods</subject><subject>High School Students</subject><subject>Historical thinking</subject><subject>History</subject><subject>History education</subject><subject>History Instruction</subject><subject>Item Analysis</subject><subject>Multiple Choice Tests</subject><subject>Protocol Analysis</subject><subject>Student Evaluation</subject><subject>Test Items</subject><subject>Thinking Skills</subject><subject>Validity</subject><issn>0737-0008</issn><issn>1532-690X</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2019</creationdate><recordtype>article</recordtype><sourceid>7SW</sourceid><recordid>eNp9kF1LwzAUhoMoOKc_YVDwujNfTVKvHEOdMvDCKd6FNB-auTUz6ZD-e1s6vfTqHHif9xx4AJggOEVQwCvICYcQiimGSEwRLUtG2REYoYLgnJXw7RiMeibvoVNwltK623CB-AighU9NiG02S8mmtLV1k7LgstWHrz99_X6dzbJXtfHGN2323OxNew5OnNoke3GYY_Byd7uaL_Ll0_3DfLbMNcWsyanlTLDKEKs5ESUqSsNFVZgKUiicNpRj5bSDipAKa6WZNcIwU1WlK4ygnIzB5XB3F8PX3qZGrsM-1t1LiZEQgmIEaUcVA6VjSClaJ3fRb1VsJYKytyN_7cjejjzY6XqToWej13-d20eEuw7u85sh97ULcau-Q9wY2ah2E6KLqtY-SfL_ix_nWXRW</recordid><startdate>20190102</startdate><enddate>20190102</enddate><creator>Smith, Mark</creator><creator>Breakstone, Joel</creator><creator>Wineburg, Sam</creator><general>Routledge</general><general>Taylor &amp; Francis Ltd</general><scope>7SW</scope><scope>BJH</scope><scope>BNH</scope><scope>BNI</scope><scope>BNJ</scope><scope>BNO</scope><scope>ERI</scope><scope>PET</scope><scope>REK</scope><scope>WWN</scope><scope>AAYXX</scope><scope>CITATION</scope><orcidid>https://orcid.org/0000-0002-1048-7786</orcidid><orcidid>https://orcid.org/0000-0003-0468-6399</orcidid><orcidid>https://orcid.org/0000-0002-4838-1522</orcidid></search><sort><creationdate>20190102</creationdate><title>History Assessments of Thinking: A Validity Study</title><author>Smith, Mark ; Breakstone, Joel ; Wineburg, Sam</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c426t-4e7686bd3ec7389159d78b5db0408fcd472afcf0a33b2cac6ed8d6dbb9f5d8473</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2019</creationdate><topic>assessment</topic><topic>Cognitive Processes</topic><topic>Educational evaluation</topic><topic>Evaluation Methods</topic><topic>High School Students</topic><topic>Historical thinking</topic><topic>History</topic><topic>History education</topic><topic>History Instruction</topic><topic>Item Analysis</topic><topic>Multiple Choice Tests</topic><topic>Protocol Analysis</topic><topic>Student Evaluation</topic><topic>Test Items</topic><topic>Thinking Skills</topic><topic>Validity</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Smith, Mark</creatorcontrib><creatorcontrib>Breakstone, Joel</creatorcontrib><creatorcontrib>Wineburg, Sam</creatorcontrib><collection>ERIC</collection><collection>ERIC (Ovid)</collection><collection>ERIC</collection><collection>ERIC</collection><collection>ERIC (Legacy Platform)</collection><collection>ERIC( SilverPlatter )</collection><collection>ERIC</collection><collection>ERIC PlusText (Legacy Platform)</collection><collection>Education Resources Information Center (ERIC)</collection><collection>ERIC</collection><collection>CrossRef</collection><jtitle>Cognition and instruction</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Smith, Mark</au><au>Breakstone, Joel</au><au>Wineburg, Sam</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><ericid>EJ1208026</ericid><atitle>History Assessments of Thinking: A Validity Study</atitle><jtitle>Cognition and instruction</jtitle><date>2019-01-02</date><risdate>2019</risdate><volume>37</volume><issue>1</issue><spage>118</spage><epage>144</epage><pages>118-144</pages><issn>0737-0008</issn><eissn>1532-690X</eissn><abstract>This article reports a validity study of History Assessments of Thinking (HATs), which are short, constructed-response assessments of historical thinking. In particular, this study focuses on aspects of cognitive validity, which is an examination of whether assessments tap the intended constructs. Think-aloud interviews with 26 high school students were used to examine the thinking elicited by 8 HATs and multiple-choice versions of these tasks. Results showed that although both HATs and multiple-choice items tapped historical thinking processes, HATs better reflected student proficiency in historical thinking than their multiple-choice counterparts. Item format also influenced the thinking elicited, with multiple-choice items eliciting more instances of construct-irrelevant reasoning than the constructed-response versions. Implications for history assessment are discussed.</abstract><cop>Philadelphia</cop><pub>Routledge</pub><doi>10.1080/07370008.2018.1499646</doi><tpages>27</tpages><orcidid>https://orcid.org/0000-0002-1048-7786</orcidid><orcidid>https://orcid.org/0000-0003-0468-6399</orcidid><orcidid>https://orcid.org/0000-0002-4838-1522</orcidid></addata></record>
fulltext fulltext
identifier ISSN: 0737-0008
ispartof Cognition and instruction, 2019-01, Vol.37 (1), p.118-144
issn 0737-0008
1532-690X
language eng
recordid cdi_eric_primary_EJ1208026
source Taylor & Francis; ERIC
subjects assessment
Cognitive Processes
Educational evaluation
Evaluation Methods
High School Students
Historical thinking
History
History education
History Instruction
Item Analysis
Multiple Choice Tests
Protocol Analysis
Student Evaluation
Test Items
Thinking Skills
Validity
title History Assessments of Thinking: A Validity Study
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-29T02%3A11%3A13IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_eric_&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=History%20Assessments%20of%20Thinking:%20A%20Validity%20Study&rft.jtitle=Cognition%20and%20instruction&rft.au=Smith,%20Mark&rft.date=2019-01-02&rft.volume=37&rft.issue=1&rft.spage=118&rft.epage=144&rft.pages=118-144&rft.issn=0737-0008&rft.eissn=1532-690X&rft_id=info:doi/10.1080/07370008.2018.1499646&rft_dat=%3Cproquest_eric_%3E2188842104%3C/proquest_eric_%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-c426t-4e7686bd3ec7389159d78b5db0408fcd472afcf0a33b2cac6ed8d6dbb9f5d8473%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_pqid=2188842104&rft_id=info:pmid/&rft_ericid=EJ1208026&rfr_iscdi=true