Cross-cultural comparability and validity of metacognitive knowledge in reading in PISA 2009: a comparison of two scoring methods

Accurate measurement of metacognitive knowledge in reading is important. Different instruments and scoring methods have been proposed but not systematically compared for their measurement comparability across cultures and validity. Given student data from 34 OECD countries in the Programme for Inter...

Full description

Saved in:
Bibliographic Details
Published in:Assessment in education : principles, policy & practice policy & practice, 2020-11, Vol.27 (6), p.635-654
Main Authors: Zhou, Ji, He, Jia, Lafontaine, Dominique
Format: Article
Language:English
Subjects:
Achievement Tests
Cross Cultural Studies
cross-cultural comparability
Discrimination
Education & enseignement
Education & instruction
Foreign Countries
International Assessment
Likert scale
Measures
Metacognition
Metacognitive knowledge
PISA
Programme for International Student Assessment (PISA)
Ratings & rankings
Reading
Reading Comprehension
Reading Motivation
Sciences sociales & comportementales, psychologie
Scoring
Secondary School Students
Social & behavioral sciences, psychology
Test Validity
Validity
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by cdi_FETCH-LOGICAL-c451t-807472a8ce20f4611a7a80553051e7ebe394fff6e2554968fd90f32a4a22d3123
cites cdi_FETCH-LOGICAL-c451t-807472a8ce20f4611a7a80553051e7ebe394fff6e2554968fd90f32a4a22d3123
container_end_page 654
container_issue 6
container_start_page 635
container_title Assessment in education : principles, policy & practice
container_volume 27
creator Zhou, Ji
He, Jia
Lafontaine, Dominique
description Accurate measurement of metacognitive knowledge in reading is important. Different instruments and scoring methods have been proposed but not systematically compared for their measurement comparability across cultures and validity. Given student data from 34 OECD countries in the Programme for International Student Assessment (PISA) in 2009, we compared two scoring methods for metacognitive knowledge in reading based on pair-wise comparisons of strategies and with conventional Likert-scale responses of selected items. Metacognitive knowledge scored with conventional Likert-scale responses demonstrated higher cross-cultural comparability than the pair-wise comparison method. Linked with reading competence, motivation and control strategy in reading, scores from the two scoring methods showed differential criterion validity, possibly related to the types of tasks (understanding and remembering versus summarising), item content (complexity and discrimination between preferred strategies in reading) and common method variance (e.g., individuals' stable response style in rating scales). Theoretical and methodological implications are discussed..
doi_str_mv 10.1080/0969594X.2020.1828820
format article
fullrecord <record><control><sourceid>proquest_eric_</sourceid><recordid>TN_cdi_eric_primary_EJ1280560</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ericid>EJ1280560</ericid><sourcerecordid>2474292575</sourcerecordid><originalsourceid>FETCH-LOGICAL-c451t-807472a8ce20f4611a7a80553051e7ebe394fff6e2554968fd90f32a4a22d3123</originalsourceid><addsrcrecordid>eNp9UcFu1DAQjSqQuhQ-oZIlzin2xE4cTlSrAq0qgdQi9WbNOnZw8caLnexqj_w5sbLliC-e8bz3ZsavKC4ZvWJU0g-0rVvR8qcroDA_SZAS6FmxYrwWZQXt06tilTFlBp0Xb1J6pvMRvF0Vf9YxpFTqyY9TRE902O4w4sZ5Nx4JDh3Zo3ddToIlWzOiDv3gRrc35NcQDt50vSFuINFg54Y-h99vH64JUNp-JHgSdCkMWWA8BJJ0iBk5i_0MXXpbvLbok3l3ui-KH59vHtdfy_tvX27X1_el5oKNpaQNbwClNkAtrxnDBiUVoqKCmcZsTNVya21tQMyL1dJ2LbUVIEeArmJQXRTVouud6Y0KcePUHlRAt8ST7xVqtTEKoJYKRCUgs94vrF0MvyeTRvUcpjjMgyrgDYcWRCNmlFhQOv9mNFbtottiPCpGVfZIvXikskfq5NHMu1x4Jjr9j3Nzx2Berc71T0vdDTbELR5C9J0a8ehDtBEH7ZKq_t_iL1W_oh4</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2474292575</pqid></control><display><type>article</type><title>Cross-cultural comparability and validity of metacognitive knowledge in reading in PISA 2009: a comparison of two scoring methods</title><source>PAIS Index</source><source>ERIC</source><source>Taylor and Francis Social Sciences and Humanities Collection</source><creator>Zhou, Ji ; He, Jia ; Lafontaine, Dominique</creator><creatorcontrib>Zhou, Ji ; He, Jia ; Lafontaine, Dominique</creatorcontrib><description>Accurate measurement of metacognitive knowledge in reading is important. Different instruments and scoring methods have been proposed but not systematically compared for their measurement comparability across cultures and validity. Given student data from 34 OECD countries in the Programme for International Student Assessment (PISA) in 2009, we compared two scoring methods for metacognitive knowledge in reading based on pair-wise comparisons of strategies and with conventional Likert-scale responses of selected items. Metacognitive knowledge scored with conventional Likert-scale responses demonstrated higher cross-cultural comparability than the pair-wise comparison method. Linked with reading competence, motivation and control strategy in reading, scores from the two scoring methods showed differential criterion validity, possibly related to the types of tasks (understanding and remembering versus summarising), item content (complexity and discrimination between preferred strategies in reading) and common method variance (e.g., individuals' stable response style in rating scales). Theoretical and methodological implications are discussed..</description><identifier>ISSN: 0969-594X</identifier><identifier>ISSN: 1465-329X</identifier><identifier>EISSN: 1465-329X</identifier><identifier>DOI: 10.1080/0969594X.2020.1828820</identifier><language>eng</language><publisher>Abingdon: Routledge</publisher><subject>Achievement Tests ; Cross Cultural Studies ; cross-cultural comparability ; Discrimination ; Education &amp; enseignement ; Education &amp; instruction ; Foreign Countries ; International Assessment ; Likert scale ; Measures ; Metacognition ; Metacognitive knowledge ; PISA ; Programme for International Student Assessment (PISA) ; Ratings &amp; rankings ; Reading ; Reading Comprehension ; Reading Motivation ; Sciences sociales &amp; comportementales, psychologie ; Scoring ; Secondary School Students ; Social &amp; behavioral sciences, psychology ; Test Validity ; Validity</subject><ispartof>Assessment in education : principles, policy &amp; practice, 2020-11, Vol.27 (6), p.635-654</ispartof><rights>2020 Informa UK Limited, trading as Taylor &amp; Francis Group 2020</rights><rights>2020 Informa UK Limited, trading as Taylor &amp; Francis Group</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c451t-807472a8ce20f4611a7a80553051e7ebe394fff6e2554968fd90f32a4a22d3123</citedby><cites>FETCH-LOGICAL-c451t-807472a8ce20f4611a7a80553051e7ebe394fff6e2554968fd90f32a4a22d3123</cites><orcidid>0000-0001-7310-4861 ; 0000-0003-1497-4634 ; 0000-0001-8318-1457</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>230,314,776,780,881,27843,27901,27902</link.rule.ids><backlink>$$Uhttp://eric.ed.gov/ERICWebPortal/detail?accno=EJ1280560$$DView record in ERIC$$Hfree_for_read</backlink></links><search><creatorcontrib>Zhou, Ji</creatorcontrib><creatorcontrib>He, Jia</creatorcontrib><creatorcontrib>Lafontaine, Dominique</creatorcontrib><title>Cross-cultural comparability and validity of metacognitive knowledge in reading in PISA 2009: a comparison of two scoring methods</title><title>Assessment in education : principles, policy &amp; practice</title><description>Accurate measurement of metacognitive knowledge in reading is important. Different instruments and scoring methods have been proposed but not systematically compared for their measurement comparability across cultures and validity. Given student data from 34 OECD countries in the Programme for International Student Assessment (PISA) in 2009, we compared two scoring methods for metacognitive knowledge in reading based on pair-wise comparisons of strategies and with conventional Likert-scale responses of selected items. Metacognitive knowledge scored with conventional Likert-scale responses demonstrated higher cross-cultural comparability than the pair-wise comparison method. Linked with reading competence, motivation and control strategy in reading, scores from the two scoring methods showed differential criterion validity, possibly related to the types of tasks (understanding and remembering versus summarising), item content (complexity and discrimination between preferred strategies in reading) and common method variance (e.g., individuals' stable response style in rating scales). Theoretical and methodological implications are discussed..</description><subject>Achievement Tests</subject><subject>Cross Cultural Studies</subject><subject>cross-cultural comparability</subject><subject>Discrimination</subject><subject>Education &amp; enseignement</subject><subject>Education &amp; instruction</subject><subject>Foreign Countries</subject><subject>International Assessment</subject><subject>Likert scale</subject><subject>Measures</subject><subject>Metacognition</subject><subject>Metacognitive knowledge</subject><subject>PISA</subject><subject>Programme for International Student Assessment (PISA)</subject><subject>Ratings &amp; rankings</subject><subject>Reading</subject><subject>Reading Comprehension</subject><subject>Reading Motivation</subject><subject>Sciences sociales &amp; comportementales, psychologie</subject><subject>Scoring</subject><subject>Secondary School Students</subject><subject>Social &amp; behavioral sciences, psychology</subject><subject>Test Validity</subject><subject>Validity</subject><issn>0969-594X</issn><issn>1465-329X</issn><issn>1465-329X</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2020</creationdate><recordtype>article</recordtype><sourceid>7SW</sourceid><sourceid>7TQ</sourceid><recordid>eNp9UcFu1DAQjSqQuhQ-oZIlzin2xE4cTlSrAq0qgdQi9WbNOnZw8caLnexqj_w5sbLliC-e8bz3ZsavKC4ZvWJU0g-0rVvR8qcroDA_SZAS6FmxYrwWZQXt06tilTFlBp0Xb1J6pvMRvF0Vf9YxpFTqyY9TRE902O4w4sZ5Nx4JDh3Zo3ddToIlWzOiDv3gRrc35NcQDt50vSFuINFg54Y-h99vH64JUNp-JHgSdCkMWWA8BJJ0iBk5i_0MXXpbvLbok3l3ui-KH59vHtdfy_tvX27X1_el5oKNpaQNbwClNkAtrxnDBiUVoqKCmcZsTNVya21tQMyL1dJ2LbUVIEeArmJQXRTVouud6Y0KcePUHlRAt8ST7xVqtTEKoJYKRCUgs94vrF0MvyeTRvUcpjjMgyrgDYcWRCNmlFhQOv9mNFbtottiPCpGVfZIvXikskfq5NHMu1x4Jjr9j3Nzx2Berc71T0vdDTbELR5C9J0a8ehDtBEH7ZKq_t_iL1W_oh4</recordid><startdate>20201101</startdate><enddate>20201101</enddate><creator>Zhou, Ji</creator><creator>He, Jia</creator><creator>Lafontaine, Dominique</creator><general>Routledge</general><general>Taylor &amp; Francis Ltd</general><scope>7SW</scope><scope>BJH</scope><scope>BNH</scope><scope>BNI</scope><scope>BNJ</scope><scope>BNO</scope><scope>ERI</scope><scope>PET</scope><scope>REK</scope><scope>WWN</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7TQ</scope><scope>DHY</scope><scope>DON</scope><scope>Q33</scope><orcidid>https://orcid.org/0000-0001-7310-4861</orcidid><orcidid>https://orcid.org/0000-0003-1497-4634</orcidid><orcidid>https://orcid.org/0000-0001-8318-1457</orcidid></search><sort><creationdate>20201101</creationdate><title>Cross-cultural comparability and validity of metacognitive knowledge in reading in PISA 2009: a comparison of two scoring methods</title><author>Zhou, Ji ; He, Jia ; Lafontaine, Dominique</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c451t-807472a8ce20f4611a7a80553051e7ebe394fff6e2554968fd90f32a4a22d3123</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2020</creationdate><topic>Achievement Tests</topic><topic>Cross Cultural Studies</topic><topic>cross-cultural comparability</topic><topic>Discrimination</topic><topic>Education &amp; enseignement</topic><topic>Education &amp; instruction</topic><topic>Foreign Countries</topic><topic>International Assessment</topic><topic>Likert scale</topic><topic>Measures</topic><topic>Metacognition</topic><topic>Metacognitive knowledge</topic><topic>PISA</topic><topic>Programme for International Student Assessment (PISA)</topic><topic>Ratings &amp; rankings</topic><topic>Reading</topic><topic>Reading Comprehension</topic><topic>Reading Motivation</topic><topic>Sciences sociales &amp; comportementales, psychologie</topic><topic>Scoring</topic><topic>Secondary School Students</topic><topic>Social &amp; behavioral sciences, psychology</topic><topic>Test Validity</topic><topic>Validity</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Zhou, Ji</creatorcontrib><creatorcontrib>He, Jia</creatorcontrib><creatorcontrib>Lafontaine, Dominique</creatorcontrib><collection>ERIC</collection><collection>ERIC (Ovid)</collection><collection>ERIC</collection><collection>ERIC</collection><collection>ERIC (Legacy Platform)</collection><collection>ERIC( SilverPlatter )</collection><collection>ERIC</collection><collection>ERIC PlusText (Legacy Platform)</collection><collection>Education Resources Information Center (ERIC)</collection><collection>ERIC</collection><collection>CrossRef</collection><collection>PAIS Index</collection><collection>PAIS International</collection><collection>PAIS International (Ovid)</collection><collection>Université de Liège - Open Repository and Bibliography (ORBI)</collection><jtitle>Assessment in education : principles, policy &amp; practice</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Zhou, Ji</au><au>He, Jia</au><au>Lafontaine, Dominique</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><ericid>EJ1280560</ericid><atitle>Cross-cultural comparability and validity of metacognitive knowledge in reading in PISA 2009: a comparison of two scoring methods</atitle><jtitle>Assessment in education : principles, policy &amp; practice</jtitle><date>2020-11-01</date><risdate>2020</risdate><volume>27</volume><issue>6</issue><spage>635</spage><epage>654</epage><pages>635-654</pages><issn>0969-594X</issn><issn>1465-329X</issn><eissn>1465-329X</eissn><abstract>Accurate measurement of metacognitive knowledge in reading is important. Different instruments and scoring methods have been proposed but not systematically compared for their measurement comparability across cultures and validity. Given student data from 34 OECD countries in the Programme for International Student Assessment (PISA) in 2009, we compared two scoring methods for metacognitive knowledge in reading based on pair-wise comparisons of strategies and with conventional Likert-scale responses of selected items. Metacognitive knowledge scored with conventional Likert-scale responses demonstrated higher cross-cultural comparability than the pair-wise comparison method. Linked with reading competence, motivation and control strategy in reading, scores from the two scoring methods showed differential criterion validity, possibly related to the types of tasks (understanding and remembering versus summarising), item content (complexity and discrimination between preferred strategies in reading) and common method variance (e.g., individuals' stable response style in rating scales). Theoretical and methodological implications are discussed..</abstract><cop>Abingdon</cop><pub>Routledge</pub><doi>10.1080/0969594X.2020.1828820</doi><tpages>20</tpages><orcidid>https://orcid.org/0000-0001-7310-4861</orcidid><orcidid>https://orcid.org/0000-0003-1497-4634</orcidid><orcidid>https://orcid.org/0000-0001-8318-1457</orcidid><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier ISSN: 0969-594X
ispartof Assessment in education : principles, policy & practice, 2020-11, Vol.27 (6), p.635-654
issn 0969-594X
1465-329X
1465-329X
language eng
recordid cdi_eric_primary_EJ1280560
source PAIS Index; ERIC; Taylor and Francis Social Sciences and Humanities Collection
subjects Achievement Tests
Cross Cultural Studies
cross-cultural comparability
Discrimination
Education & enseignement
Education & instruction
Foreign Countries
International Assessment
Likert scale
Measures
Metacognition
Metacognitive knowledge
PISA
Programme for International Student Assessment (PISA)
Ratings & rankings
Reading
Reading Comprehension
Reading Motivation
Sciences sociales & comportementales, psychologie
Scoring
Secondary School Students
Social & behavioral sciences, psychology
Test Validity
Validity
title Cross-cultural comparability and validity of metacognitive knowledge in reading in PISA 2009: a comparison of two scoring methods
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-02-08T05%3A30%3A12IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_eric_&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Cross-cultural%20comparability%20and%20validity%20of%20metacognitive%20knowledge%20in%20reading%20in%20PISA%202009:%20a%20comparison%20of%20two%20scoring%20methods&rft.jtitle=Assessment%20in%20education%20:%20principles,%20policy%20&%20practice&rft.au=Zhou,%20Ji&rft.date=2020-11-01&rft.volume=27&rft.issue=6&rft.spage=635&rft.epage=654&rft.pages=635-654&rft.issn=0969-594X&rft.eissn=1465-329X&rft_id=info:doi/10.1080/0969594X.2020.1828820&rft_dat=%3Cproquest_eric_%3E2474292575%3C/proquest_eric_%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-c451t-807472a8ce20f4611a7a80553051e7ebe394fff6e2554968fd90f32a4a22d3123%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_pqid=2474292575&rft_id=info:pmid/&rft_ericid=EJ1280560&rfr_iscdi=true