Sketched covariance testing: A compression-statistics tradeoff

Hypothesis testing of covariance matrices is an important problem in multivariate analysis. Given n data samples and a covariance matrix ∑ 0 , the goal is to determine whether or not the data is consistent with this matrix. In this paper we introduce a framework that we call sketched covariance test...

Full description

Saved in:
Bibliographic Details
Main Authors: Dasarathy, Gautam, Shah, Parikshit, Baraniuk, Richard G.
Format: Conference Proceeding
Language:English
Subjects:
Complexity theory
Covariance matrices
Estimation
Proteins
Sparse matrices
Testing
Upper bound
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by
cites
container_end_page 680
container_issue
container_start_page 676
container_title
container_volume
creator Dasarathy, Gautam
Shah, Parikshit
Baraniuk, Richard G.
description Hypothesis testing of covariance matrices is an important problem in multivariate analysis. Given n data samples and a covariance matrix ∑ 0 , the goal is to determine whether or not the data is consistent with this matrix. In this paper we introduce a framework that we call sketched covariance testing, where the data is provided after being compressed by multiplying by a "sketching" matrix A chosen by the analyst. We propose a statistical test in this setting and quantify an achievable sample complexity as a function of the amount of compression. Our result reveals an intriguing achievable tradeoff between the compression ratio and the statistical information required for reliable hypothesis testing; the sample complexity increases as the fourth power of the amount of compression.
doi_str_mv 10.1109/ACSSC.2017.8335428
format conference_proceeding
fullrecord <record><control><sourceid>ieee_CHZPO</sourceid><recordid>TN_cdi_ieee_primary_8335428</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>8335428</ieee_id><sourcerecordid>8335428</sourcerecordid><originalsourceid>FETCH-LOGICAL-i175t-1ffeb096199abbdab873d26d8a7cab891cdf52022c09881bf348beef78dc5c513</originalsourceid><addsrcrecordid>eNotj9tKAzEYhKMgWNu-gN7sC-yaQ5P88UJYFk9Q8GLrdcnhj0btbtkEwbd3wV4NMx8MM4RcM9owRs1t2_V913DKdANCyA2HM7I2GpgUoKhSSp-TBZda1VxQcUmucv6klFMOfEHu-y8s_gND5ccfOyU7eKwK5pKG97uqndPDccKc0zjUudiSZuJzVSYbcIxxRS6i_c64PumSvD0-7Lrnevv69NK12zoxLUvNYkRHjWLGWOeCdaBF4CqA1X42hvkQ5byIe2oAmItiAw4xagheesnEktz89yZE3B-ndLDT7_50V_wBEu1KiA</addsrcrecordid><sourcetype>Publisher</sourcetype><iscdi>true</iscdi><recordtype>conference_proceeding</recordtype></control><display><type>conference_proceeding</type><title>Sketched covariance testing: A compression-statistics tradeoff</title><source>IEEE Xplore All Conference Series</source><creator>Dasarathy, Gautam ; Shah, Parikshit ; Baraniuk, Richard G.</creator><creatorcontrib>Dasarathy, Gautam ; Shah, Parikshit ; Baraniuk, Richard G.</creatorcontrib><description>Hypothesis testing of covariance matrices is an important problem in multivariate analysis. Given n data samples and a covariance matrix ∑ 0 , the goal is to determine whether or not the data is consistent with this matrix. In this paper we introduce a framework that we call sketched covariance testing, where the data is provided after being compressed by multiplying by a "sketching" matrix A chosen by the analyst. We propose a statistical test in this setting and quantify an achievable sample complexity as a function of the amount of compression. Our result reveals an intriguing achievable tradeoff between the compression ratio and the statistical information required for reliable hypothesis testing; the sample complexity increases as the fourth power of the amount of compression.</description><identifier>EISSN: 2576-2303</identifier><identifier>EISBN: 9781538606667</identifier><identifier>EISBN: 9781538618233</identifier><identifier>EISBN: 1538606666</identifier><identifier>EISBN: 1538618230</identifier><identifier>DOI: 10.1109/ACSSC.2017.8335428</identifier><language>eng</language><publisher>IEEE</publisher><subject>Complexity theory ; Covariance matrices ; Estimation ; Proteins ; Sparse matrices ; Testing ; Upper bound</subject><ispartof>2017 51st Asilomar Conference on Signals, Systems, and Computers, 2017, p.676-680</ispartof><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/8335428$$EHTML$$P50$$Gieee$$H</linktohtml><link.rule.ids>309,310,780,784,789,790,27925,54555,54932</link.rule.ids><linktorsrc>$$Uhttps://ieeexplore.ieee.org/document/8335428$$EView_record_in_IEEE$$FView_record_in_$$GIEEE</linktorsrc></links><search><creatorcontrib>Dasarathy, Gautam</creatorcontrib><creatorcontrib>Shah, Parikshit</creatorcontrib><creatorcontrib>Baraniuk, Richard G.</creatorcontrib><title>Sketched covariance testing: A compression-statistics tradeoff</title><title>2017 51st Asilomar Conference on Signals, Systems, and Computers</title><addtitle>ACSSC</addtitle><description>Hypothesis testing of covariance matrices is an important problem in multivariate analysis. Given n data samples and a covariance matrix ∑ 0 , the goal is to determine whether or not the data is consistent with this matrix. In this paper we introduce a framework that we call sketched covariance testing, where the data is provided after being compressed by multiplying by a "sketching" matrix A chosen by the analyst. We propose a statistical test in this setting and quantify an achievable sample complexity as a function of the amount of compression. Our result reveals an intriguing achievable tradeoff between the compression ratio and the statistical information required for reliable hypothesis testing; the sample complexity increases as the fourth power of the amount of compression.</description><subject>Complexity theory</subject><subject>Covariance matrices</subject><subject>Estimation</subject><subject>Proteins</subject><subject>Sparse matrices</subject><subject>Testing</subject><subject>Upper bound</subject><issn>2576-2303</issn><isbn>9781538606667</isbn><isbn>9781538618233</isbn><isbn>1538606666</isbn><isbn>1538618230</isbn><fulltext>true</fulltext><rsrctype>conference_proceeding</rsrctype><creationdate>2017</creationdate><recordtype>conference_proceeding</recordtype><sourceid>6IE</sourceid><recordid>eNotj9tKAzEYhKMgWNu-gN7sC-yaQ5P88UJYFk9Q8GLrdcnhj0btbtkEwbd3wV4NMx8MM4RcM9owRs1t2_V913DKdANCyA2HM7I2GpgUoKhSSp-TBZda1VxQcUmucv6klFMOfEHu-y8s_gND5ccfOyU7eKwK5pKG97uqndPDccKc0zjUudiSZuJzVSYbcIxxRS6i_c64PumSvD0-7Lrnevv69NK12zoxLUvNYkRHjWLGWOeCdaBF4CqA1X42hvkQ5byIe2oAmItiAw4xagheesnEktz89yZE3B-ndLDT7_50V_wBEu1KiA</recordid><startdate>201710</startdate><enddate>201710</enddate><creator>Dasarathy, Gautam</creator><creator>Shah, Parikshit</creator><creator>Baraniuk, Richard G.</creator><general>IEEE</general><scope>6IE</scope><scope>6IH</scope><scope>CBEJK</scope><scope>RIE</scope><scope>RIO</scope></search><sort><creationdate>201710</creationdate><title>Sketched covariance testing: A compression-statistics tradeoff</title><author>Dasarathy, Gautam ; Shah, Parikshit ; Baraniuk, Richard G.</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-i175t-1ffeb096199abbdab873d26d8a7cab891cdf52022c09881bf348beef78dc5c513</frbrgroupid><rsrctype>conference_proceedings</rsrctype><prefilter>conference_proceedings</prefilter><language>eng</language><creationdate>2017</creationdate><topic>Complexity theory</topic><topic>Covariance matrices</topic><topic>Estimation</topic><topic>Proteins</topic><topic>Sparse matrices</topic><topic>Testing</topic><topic>Upper bound</topic><toplevel>online_resources</toplevel><creatorcontrib>Dasarathy, Gautam</creatorcontrib><creatorcontrib>Shah, Parikshit</creatorcontrib><creatorcontrib>Baraniuk, Richard G.</creatorcontrib><collection>IEEE Electronic Library (IEL) Conference Proceedings</collection><collection>IEEE Proceedings Order Plan (POP) 1998-present by volume</collection><collection>IEEE Xplore All Conference Proceedings</collection><collection>IEEE</collection><collection>IEEE Proceedings Order Plans (POP) 1998-present</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Dasarathy, Gautam</au><au>Shah, Parikshit</au><au>Baraniuk, Richard G.</au><format>book</format><genre>proceeding</genre><ristype>CONF</ristype><atitle>Sketched covariance testing: A compression-statistics tradeoff</atitle><btitle>2017 51st Asilomar Conference on Signals, Systems, and Computers</btitle><stitle>ACSSC</stitle><date>2017-10</date><risdate>2017</risdate><spage>676</spage><epage>680</epage><pages>676-680</pages><eissn>2576-2303</eissn><eisbn>9781538606667</eisbn><eisbn>9781538618233</eisbn><eisbn>1538606666</eisbn><eisbn>1538618230</eisbn><abstract>Hypothesis testing of covariance matrices is an important problem in multivariate analysis. Given n data samples and a covariance matrix ∑ 0 , the goal is to determine whether or not the data is consistent with this matrix. In this paper we introduce a framework that we call sketched covariance testing, where the data is provided after being compressed by multiplying by a "sketching" matrix A chosen by the analyst. We propose a statistical test in this setting and quantify an achievable sample complexity as a function of the amount of compression. Our result reveals an intriguing achievable tradeoff between the compression ratio and the statistical information required for reliable hypothesis testing; the sample complexity increases as the fourth power of the amount of compression.</abstract><pub>IEEE</pub><doi>10.1109/ACSSC.2017.8335428</doi><tpages>5</tpages></addata></record>
fulltext fulltext_linktorsrc
identifier EISSN: 2576-2303
ispartof 2017 51st Asilomar Conference on Signals, Systems, and Computers, 2017, p.676-680
issn 2576-2303
language eng
recordid cdi_ieee_primary_8335428
source IEEE Xplore All Conference Series
subjects Complexity theory
Covariance matrices
Estimation
Proteins
Sparse matrices
Testing
Upper bound
title Sketched covariance testing: A compression-statistics tradeoff
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-26T23%3A28%3A31IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-ieee_CHZPO&rft_val_fmt=info:ofi/fmt:kev:mtx:book&rft.genre=proceeding&rft.atitle=Sketched%20covariance%20testing:%20A%20compression-statistics%20tradeoff&rft.btitle=2017%2051st%20Asilomar%20Conference%20on%20Signals,%20Systems,%20and%20Computers&rft.au=Dasarathy,%20Gautam&rft.date=2017-10&rft.spage=676&rft.epage=680&rft.pages=676-680&rft.eissn=2576-2303&rft_id=info:doi/10.1109/ACSSC.2017.8335428&rft.eisbn=9781538606667&rft.eisbn_list=9781538618233&rft.eisbn_list=1538606666&rft.eisbn_list=1538618230&rft_dat=%3Cieee_CHZPO%3E8335428%3C/ieee_CHZPO%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-i175t-1ffeb096199abbdab873d26d8a7cab891cdf52022c09881bf348beef78dc5c513%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_id=info:pmid/&rft_ieee_id=8335428&rfr_iscdi=true