On boosting the power of Chatterjee’s rank correlation

Summary The ingenious approach of Chatterjee (2021) to estimate a measure of dependence first proposed by Dette et al. (2013) based on simple rank statistics has quickly caught attention. This measure of dependence has the appealing property of being between 0 and 1, and being 0 or 1 if and only if...

Full description

Saved in:
Bibliographic Details
Published in:Biometrika 2023-06, Vol.110 (2), p.283-299
Main Authors: Lin, Z, Han, F
Format: Article
Language:English
Subjects:
Correlation coefficient
Correlation coefficients
Independent variables
Mathematical analysis
Random variables
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by cdi_FETCH-LOGICAL-c301t-1fc0a15817854887580fe0aad74b2f3e4964ddf666a10336b3dcd0e279f301173
cites cdi_FETCH-LOGICAL-c301t-1fc0a15817854887580fe0aad74b2f3e4964ddf666a10336b3dcd0e279f301173
container_end_page 299
container_issue 2
container_start_page 283
container_title Biometrika
container_volume 110
creator Lin, Z
Han, F
description Summary The ingenious approach of Chatterjee (2021) to estimate a measure of dependence first proposed by Dette et al. (2013) based on simple rank statistics has quickly caught attention. This measure of dependence has the appealing property of being between 0 and 1, and being 0 or 1 if and only if the corresponding pair of random variables is independent or one is a measurable function of the other almost surely. However, more recent studies (Cao & Bickel 2020; Shi et al. 2022b) showed that independence tests based on Chatterjee’s rank correlation are unfortunately rate inefficient against various local alternatives and they call for variants. We answer this call by proposing an improvement to Chatterjee’s rank correlation that still consistently estimates the same dependence measure, but provably achieves near-parametric efficiency in testing against Gaussian rotation alternatives. This is possible by incorporating many right nearest neighbours in constructing the correlation coefficients. We thus overcome the ‘ only one disadvantage’ of Chatterjee’s rank correlation (Chatterjee, 2021, § 7).
doi_str_mv 10.1093/biomet/asac048
format article
fullrecord <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_journals_3049131810</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><oup_id>10.1093/biomet/asac048</oup_id><sourcerecordid>3049131810</sourcerecordid><originalsourceid>FETCH-LOGICAL-c301t-1fc0a15817854887580fe0aad74b2f3e4964ddf666a10336b3dcd0e279f301173</originalsourceid><addsrcrecordid>eNqFkL1OwzAUhS0EEiWwMltiYkh7b-04zogi_qRKXWC2nMSmKW0cbEeIjdfg9XgSUqU709GVzneP9BFyjTBHKNiiat3exIUOugYuT8gMueApyxBOyQwARMo45-fkIoTt4RSZmBG57mjlXIht90bjxtDefRpPnaXlRsdo_NaY3--fQL3u3mntvDc7HVvXXZIzq3fBXB0zIa8P9y_lU7paPz6Xd6u0ZoAxRVuDxkxiLjMuZZ5JsAa0bnJeLS0zvBC8aawQQiMwJirW1A2YZV7YkcecJeRm-tt79zGYENXWDb4bJxUDXiBDOYIJmU-t2rsQvLGq9-1e-y-FoA521GRHHe2MwO0EuKH_r_sHVTVoJg</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>3049131810</pqid></control><display><type>article</type><title>On boosting the power of Chatterjee’s rank correlation</title><source>Oxford Journals Online</source><creator>Lin, Z ; Han, F</creator><creatorcontrib>Lin, Z ; Han, F</creatorcontrib><description>Summary The ingenious approach of Chatterjee (2021) to estimate a measure of dependence first proposed by Dette et al. (2013) based on simple rank statistics has quickly caught attention. This measure of dependence has the appealing property of being between 0 and 1, and being 0 or 1 if and only if the corresponding pair of random variables is independent or one is a measurable function of the other almost surely. However, more recent studies (Cao &amp; Bickel 2020; Shi et al. 2022b) showed that independence tests based on Chatterjee’s rank correlation are unfortunately rate inefficient against various local alternatives and they call for variants. We answer this call by proposing an improvement to Chatterjee’s rank correlation that still consistently estimates the same dependence measure, but provably achieves near-parametric efficiency in testing against Gaussian rotation alternatives. This is possible by incorporating many right nearest neighbours in constructing the correlation coefficients. We thus overcome the ‘ only one disadvantage’ of Chatterjee’s rank correlation (Chatterjee, 2021, § 7).</description><identifier>ISSN: 0006-3444</identifier><identifier>EISSN: 1464-3510</identifier><identifier>DOI: 10.1093/biomet/asac048</identifier><language>eng</language><publisher>Oxford: Oxford University Press</publisher><subject>Correlation coefficient ; Correlation coefficients ; Independent variables ; Mathematical analysis ; Random variables</subject><ispartof>Biometrika, 2023-06, Vol.110 (2), p.283-299</ispartof><rights>The Author(s) 2022. Published by Oxford University Press on behalf of the Biometrika Trust. All rights reserved. For permissions, please email: journals.permissions@oup.com 2022</rights><rights>The Author(s) 2022. Published by Oxford University Press on behalf of the Biometrika Trust. All rights reserved. For permissions, please email: journals.permissions@oup.com</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c301t-1fc0a15817854887580fe0aad74b2f3e4964ddf666a10336b3dcd0e279f301173</citedby><cites>FETCH-LOGICAL-c301t-1fc0a15817854887580fe0aad74b2f3e4964ddf666a10336b3dcd0e279f301173</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>314,780,784,27924,27925</link.rule.ids></links><search><creatorcontrib>Lin, Z</creatorcontrib><creatorcontrib>Han, F</creatorcontrib><title>On boosting the power of Chatterjee’s rank correlation</title><title>Biometrika</title><description>Summary The ingenious approach of Chatterjee (2021) to estimate a measure of dependence first proposed by Dette et al. (2013) based on simple rank statistics has quickly caught attention. This measure of dependence has the appealing property of being between 0 and 1, and being 0 or 1 if and only if the corresponding pair of random variables is independent or one is a measurable function of the other almost surely. However, more recent studies (Cao &amp; Bickel 2020; Shi et al. 2022b) showed that independence tests based on Chatterjee’s rank correlation are unfortunately rate inefficient against various local alternatives and they call for variants. We answer this call by proposing an improvement to Chatterjee’s rank correlation that still consistently estimates the same dependence measure, but provably achieves near-parametric efficiency in testing against Gaussian rotation alternatives. This is possible by incorporating many right nearest neighbours in constructing the correlation coefficients. We thus overcome the ‘ only one disadvantage’ of Chatterjee’s rank correlation (Chatterjee, 2021, § 7).</description><subject>Correlation coefficient</subject><subject>Correlation coefficients</subject><subject>Independent variables</subject><subject>Mathematical analysis</subject><subject>Random variables</subject><issn>0006-3444</issn><issn>1464-3510</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2023</creationdate><recordtype>article</recordtype><recordid>eNqFkL1OwzAUhS0EEiWwMltiYkh7b-04zogi_qRKXWC2nMSmKW0cbEeIjdfg9XgSUqU709GVzneP9BFyjTBHKNiiat3exIUOugYuT8gMueApyxBOyQwARMo45-fkIoTt4RSZmBG57mjlXIht90bjxtDefRpPnaXlRsdo_NaY3--fQL3u3mntvDc7HVvXXZIzq3fBXB0zIa8P9y_lU7paPz6Xd6u0ZoAxRVuDxkxiLjMuZZ5JsAa0bnJeLS0zvBC8aawQQiMwJirW1A2YZV7YkcecJeRm-tt79zGYENXWDb4bJxUDXiBDOYIJmU-t2rsQvLGq9-1e-y-FoA521GRHHe2MwO0EuKH_r_sHVTVoJg</recordid><startdate>20230601</startdate><enddate>20230601</enddate><creator>Lin, Z</creator><creator>Han, F</creator><general>Oxford University Press</general><general>Oxford Publishing Limited (England)</general><scope>AAYXX</scope><scope>CITATION</scope><scope>7QO</scope><scope>8FD</scope><scope>FR3</scope><scope>P64</scope></search><sort><creationdate>20230601</creationdate><title>On boosting the power of Chatterjee’s rank correlation</title><author>Lin, Z ; Han, F</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c301t-1fc0a15817854887580fe0aad74b2f3e4964ddf666a10336b3dcd0e279f301173</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2023</creationdate><topic>Correlation coefficient</topic><topic>Correlation coefficients</topic><topic>Independent variables</topic><topic>Mathematical analysis</topic><topic>Random variables</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Lin, Z</creatorcontrib><creatorcontrib>Han, F</creatorcontrib><collection>CrossRef</collection><collection>Biotechnology Research Abstracts</collection><collection>Technology Research Database</collection><collection>Engineering Research Database</collection><collection>Biotechnology and BioEngineering Abstracts</collection><jtitle>Biometrika</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Lin, Z</au><au>Han, F</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>On boosting the power of Chatterjee’s rank correlation</atitle><jtitle>Biometrika</jtitle><date>2023-06-01</date><risdate>2023</risdate><volume>110</volume><issue>2</issue><spage>283</spage><epage>299</epage><pages>283-299</pages><issn>0006-3444</issn><eissn>1464-3510</eissn><abstract>Summary The ingenious approach of Chatterjee (2021) to estimate a measure of dependence first proposed by Dette et al. (2013) based on simple rank statistics has quickly caught attention. This measure of dependence has the appealing property of being between 0 and 1, and being 0 or 1 if and only if the corresponding pair of random variables is independent or one is a measurable function of the other almost surely. However, more recent studies (Cao &amp; Bickel 2020; Shi et al. 2022b) showed that independence tests based on Chatterjee’s rank correlation are unfortunately rate inefficient against various local alternatives and they call for variants. We answer this call by proposing an improvement to Chatterjee’s rank correlation that still consistently estimates the same dependence measure, but provably achieves near-parametric efficiency in testing against Gaussian rotation alternatives. This is possible by incorporating many right nearest neighbours in constructing the correlation coefficients. We thus overcome the ‘ only one disadvantage’ of Chatterjee’s rank correlation (Chatterjee, 2021, § 7).</abstract><cop>Oxford</cop><pub>Oxford University Press</pub><doi>10.1093/biomet/asac048</doi><tpages>17</tpages></addata></record>
fulltext fulltext
identifier ISSN: 0006-3444
ispartof Biometrika, 2023-06, Vol.110 (2), p.283-299
issn 0006-3444
1464-3510
language eng
recordid cdi_proquest_journals_3049131810
source Oxford Journals Online
subjects Correlation coefficient
Correlation coefficients
Independent variables
Mathematical analysis
Random variables
title On boosting the power of Chatterjee’s rank correlation
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-22T22%3A07%3A06IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=On%20boosting%20the%20power%20of%20Chatterjee%E2%80%99s%20rank%20correlation&rft.jtitle=Biometrika&rft.au=Lin,%20Z&rft.date=2023-06-01&rft.volume=110&rft.issue=2&rft.spage=283&rft.epage=299&rft.pages=283-299&rft.issn=0006-3444&rft.eissn=1464-3510&rft_id=info:doi/10.1093/biomet/asac048&rft_dat=%3Cproquest_cross%3E3049131810%3C/proquest_cross%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-c301t-1fc0a15817854887580fe0aad74b2f3e4964ddf666a10336b3dcd0e279f301173%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_pqid=3049131810&rft_id=info:pmid/&rft_oup_id=10.1093/biomet/asac048&rfr_iscdi=true