Loading…
Deep Least Squares Fisher Discriminant Analysis
While being one of the first and most elegant tools for dimensionality reduction, Fisher linear discriminant analysis (FLDA) is not currently considered among the top methods for feature extraction or classification. In this paper, we will review two recent approaches to FLDA, namely, least squares...
Saved in:
Published in: | IEEE transaction on neural networks and learning systems 2020-08, Vol.31 (8), p.2752-2763 |
---|---|
Main Authors: | , |
Format: | Article |
Language: | English |
Subjects: | |
Citations: | Items that this one cites Items that cite this one |
Online Access: | Get full text |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
cited_by | cdi_FETCH-LOGICAL-c395t-11d1abdd9a911bea89a2312e71dacfd3ee6ded908bc34e28d94c0ef1f952dd793 |
---|---|
cites | cdi_FETCH-LOGICAL-c395t-11d1abdd9a911bea89a2312e71dacfd3ee6ded908bc34e28d94c0ef1f952dd793 |
container_end_page | 2763 |
container_issue | 8 |
container_start_page | 2752 |
container_title | IEEE transaction on neural networks and learning systems |
container_volume | 31 |
creator | Diaz-Vico, David Dorronsoro, Jose R. |
description | While being one of the first and most elegant tools for dimensionality reduction, Fisher linear discriminant analysis (FLDA) is not currently considered among the top methods for feature extraction or classification. In this paper, we will review two recent approaches to FLDA, namely, least squares Fisher discriminant analysis (LSFDA) and regularized kernel FDA (RKFDA) and propose deep FDA (DFDA), a straightforward nonlinear extension of LSFDA that takes advantage of the recent advances on deep neural networks. We will compare the performance of RKFDA and DFDA on a large number of two-class and multiclass problems, many of them involving class-imbalanced data sets and some having quite large sample sizes; we will use, for this, the areas under the receiver operating characteristics (ROCs) curve of the classifiers considered. As we shall see, the classification performance of both methods is often very similar and particularly good on imbalanced problems, but building DFDA models is considerably much faster than doing so for RKFDA, particularly in problems with quite large sample sizes. |
doi_str_mv | 10.1109/TNNLS.2019.2906302 |
format | article |
fullrecord | <record><control><sourceid>proquest_ieee_</sourceid><recordid>TN_cdi_proquest_miscellaneous_2210272674</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>8688630</ieee_id><sourcerecordid>2210272674</sourcerecordid><originalsourceid>FETCH-LOGICAL-c395t-11d1abdd9a911bea89a2312e71dacfd3ee6ded908bc34e28d94c0ef1f952dd793</originalsourceid><addsrcrecordid>eNpdkMFOwkAQhjdGIwR5AU1MEy9eCjuzpd09EhA1afAAJt6abXcaS0oL3fbA27sIcnAvs8l8_2TmY-we-AiAq_F6uYxXI-SgRqh4KDhesT5CiD4KKa8v_-irx4bWbrh7IZ-EgbplPcGV4kEQ9dl4TrTzYtK29Vb7TjdkvUVhv6nx5oXNmmJbVLpqvWmly4Mt7B27yXVpaXiuA_a5eFnP3vz44_V9No39TKhJ6wMY0KkxSiuAlLRUGgUgRWB0lhtBFBoyiss0EwGhNCrIOOWQqwkaEykxYM-nubum3ndk22Tr1qGy1BXVnU0QgWOEYRQ49Okfuqm7xu3rqEBAxBEQHYUnKmtqaxvKk507TjeHBHhyNJr8Gk2ORpOzURd6PI_u0i2ZS-TPnwMeTkBBRJe2DKV0efEDeJV46Q</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2431702122</pqid></control><display><type>article</type><title>Deep Least Squares Fisher Discriminant Analysis</title><source>IEEE Xplore (Online service)</source><creator>Diaz-Vico, David ; Dorronsoro, Jose R.</creator><creatorcontrib>Diaz-Vico, David ; Dorronsoro, Jose R.</creatorcontrib><description>While being one of the first and most elegant tools for dimensionality reduction, Fisher linear discriminant analysis (FLDA) is not currently considered among the top methods for feature extraction or classification. In this paper, we will review two recent approaches to FLDA, namely, least squares Fisher discriminant analysis (LSFDA) and regularized kernel FDA (RKFDA) and propose deep FDA (DFDA), a straightforward nonlinear extension of LSFDA that takes advantage of the recent advances on deep neural networks. We will compare the performance of RKFDA and DFDA on a large number of two-class and multiclass problems, many of them involving class-imbalanced data sets and some having quite large sample sizes; we will use, for this, the areas under the receiver operating characteristics (ROCs) curve of the classifiers considered. As we shall see, the classification performance of both methods is often very similar and particularly good on imbalanced problems, but building DFDA models is considerably much faster than doing so for RKFDA, particularly in problems with quite large sample sizes.</description><identifier>ISSN: 2162-237X</identifier><identifier>EISSN: 2162-2388</identifier><identifier>DOI: 10.1109/TNNLS.2019.2906302</identifier><identifier>PMID: 30990447</identifier><identifier>CODEN: ITNNAL</identifier><language>eng</language><publisher>United States: IEEE</publisher><subject>Artificial neural networks ; Classification ; Covariance matrices ; Deep neural networks (DNNs) ; Discriminant analysis ; Encoding ; Feature extraction ; Fisher discriminant analysis (FDA) ; Kernel ; kernel discriminant analysis ; Least squares ; Linear discriminant analysis ; Neural networks ; nonlinear classifiers ; Proposals ; Training</subject><ispartof>IEEE transaction on neural networks and learning systems, 2020-08, Vol.31 (8), p.2752-2763</ispartof><rights>Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2020</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c395t-11d1abdd9a911bea89a2312e71dacfd3ee6ded908bc34e28d94c0ef1f952dd793</citedby><cites>FETCH-LOGICAL-c395t-11d1abdd9a911bea89a2312e71dacfd3ee6ded908bc34e28d94c0ef1f952dd793</cites><orcidid>0000-0002-5271-0616 ; 0000-0002-4002-5312</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/8688630$$EHTML$$P50$$Gieee$$H</linktohtml><link.rule.ids>314,780,784,27924,27925,54796</link.rule.ids><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/30990447$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><creatorcontrib>Diaz-Vico, David</creatorcontrib><creatorcontrib>Dorronsoro, Jose R.</creatorcontrib><title>Deep Least Squares Fisher Discriminant Analysis</title><title>IEEE transaction on neural networks and learning systems</title><addtitle>TNNLS</addtitle><addtitle>IEEE Trans Neural Netw Learn Syst</addtitle><description>While being one of the first and most elegant tools for dimensionality reduction, Fisher linear discriminant analysis (FLDA) is not currently considered among the top methods for feature extraction or classification. In this paper, we will review two recent approaches to FLDA, namely, least squares Fisher discriminant analysis (LSFDA) and regularized kernel FDA (RKFDA) and propose deep FDA (DFDA), a straightforward nonlinear extension of LSFDA that takes advantage of the recent advances on deep neural networks. We will compare the performance of RKFDA and DFDA on a large number of two-class and multiclass problems, many of them involving class-imbalanced data sets and some having quite large sample sizes; we will use, for this, the areas under the receiver operating characteristics (ROCs) curve of the classifiers considered. As we shall see, the classification performance of both methods is often very similar and particularly good on imbalanced problems, but building DFDA models is considerably much faster than doing so for RKFDA, particularly in problems with quite large sample sizes.</description><subject>Artificial neural networks</subject><subject>Classification</subject><subject>Covariance matrices</subject><subject>Deep neural networks (DNNs)</subject><subject>Discriminant analysis</subject><subject>Encoding</subject><subject>Feature extraction</subject><subject>Fisher discriminant analysis (FDA)</subject><subject>Kernel</subject><subject>kernel discriminant analysis</subject><subject>Least squares</subject><subject>Linear discriminant analysis</subject><subject>Neural networks</subject><subject>nonlinear classifiers</subject><subject>Proposals</subject><subject>Training</subject><issn>2162-237X</issn><issn>2162-2388</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2020</creationdate><recordtype>article</recordtype><recordid>eNpdkMFOwkAQhjdGIwR5AU1MEy9eCjuzpd09EhA1afAAJt6abXcaS0oL3fbA27sIcnAvs8l8_2TmY-we-AiAq_F6uYxXI-SgRqh4KDhesT5CiD4KKa8v_-irx4bWbrh7IZ-EgbplPcGV4kEQ9dl4TrTzYtK29Vb7TjdkvUVhv6nx5oXNmmJbVLpqvWmly4Mt7B27yXVpaXiuA_a5eFnP3vz44_V9No39TKhJ6wMY0KkxSiuAlLRUGgUgRWB0lhtBFBoyiss0EwGhNCrIOOWQqwkaEykxYM-nubum3ndk22Tr1qGy1BXVnU0QgWOEYRQ49Okfuqm7xu3rqEBAxBEQHYUnKmtqaxvKk507TjeHBHhyNJr8Gk2ORpOzURd6PI_u0i2ZS-TPnwMeTkBBRJe2DKV0efEDeJV46Q</recordid><startdate>20200801</startdate><enddate>20200801</enddate><creator>Diaz-Vico, David</creator><creator>Dorronsoro, Jose R.</creator><general>IEEE</general><general>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</general><scope>97E</scope><scope>RIA</scope><scope>RIE</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7QF</scope><scope>7QO</scope><scope>7QP</scope><scope>7QQ</scope><scope>7QR</scope><scope>7SC</scope><scope>7SE</scope><scope>7SP</scope><scope>7SR</scope><scope>7TA</scope><scope>7TB</scope><scope>7TK</scope><scope>7U5</scope><scope>8BQ</scope><scope>8FD</scope><scope>F28</scope><scope>FR3</scope><scope>H8D</scope><scope>JG9</scope><scope>JQ2</scope><scope>KR7</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><scope>P64</scope><scope>7X8</scope><orcidid>https://orcid.org/0000-0002-5271-0616</orcidid><orcidid>https://orcid.org/0000-0002-4002-5312</orcidid></search><sort><creationdate>20200801</creationdate><title>Deep Least Squares Fisher Discriminant Analysis</title><author>Diaz-Vico, David ; Dorronsoro, Jose R.</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c395t-11d1abdd9a911bea89a2312e71dacfd3ee6ded908bc34e28d94c0ef1f952dd793</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2020</creationdate><topic>Artificial neural networks</topic><topic>Classification</topic><topic>Covariance matrices</topic><topic>Deep neural networks (DNNs)</topic><topic>Discriminant analysis</topic><topic>Encoding</topic><topic>Feature extraction</topic><topic>Fisher discriminant analysis (FDA)</topic><topic>Kernel</topic><topic>kernel discriminant analysis</topic><topic>Least squares</topic><topic>Linear discriminant analysis</topic><topic>Neural networks</topic><topic>nonlinear classifiers</topic><topic>Proposals</topic><topic>Training</topic><toplevel>online_resources</toplevel><creatorcontrib>Diaz-Vico, David</creatorcontrib><creatorcontrib>Dorronsoro, Jose R.</creatorcontrib><collection>IEEE All-Society Periodicals Package (ASPP) 2005-present</collection><collection>IEEE All-Society Periodicals Package (ASPP) 1998–Present</collection><collection>IEEE/IET Electronic Library (IEL)</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>Aluminium Industry Abstracts</collection><collection>Biotechnology Research Abstracts</collection><collection>Calcium & Calcified Tissue Abstracts</collection><collection>Ceramic Abstracts</collection><collection>Chemoreception Abstracts</collection><collection>Computer and Information Systems Abstracts</collection><collection>Corrosion Abstracts</collection><collection>Electronics & Communications Abstracts</collection><collection>Engineered Materials Abstracts</collection><collection>Materials Business File</collection><collection>Mechanical & Transportation Engineering Abstracts</collection><collection>Neurosciences Abstracts</collection><collection>Solid State and Superconductivity Abstracts</collection><collection>METADEX</collection><collection>Technology Research Database</collection><collection>ANTE: Abstracts in New Technology & Engineering</collection><collection>Engineering Research Database</collection><collection>Aerospace Database</collection><collection>Materials Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Civil Engineering Abstracts</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><collection>Biotechnology and BioEngineering Abstracts</collection><collection>MEDLINE - Academic</collection><jtitle>IEEE transaction on neural networks and learning systems</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Diaz-Vico, David</au><au>Dorronsoro, Jose R.</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Deep Least Squares Fisher Discriminant Analysis</atitle><jtitle>IEEE transaction on neural networks and learning systems</jtitle><stitle>TNNLS</stitle><addtitle>IEEE Trans Neural Netw Learn Syst</addtitle><date>2020-08-01</date><risdate>2020</risdate><volume>31</volume><issue>8</issue><spage>2752</spage><epage>2763</epage><pages>2752-2763</pages><issn>2162-237X</issn><eissn>2162-2388</eissn><coden>ITNNAL</coden><abstract>While being one of the first and most elegant tools for dimensionality reduction, Fisher linear discriminant analysis (FLDA) is not currently considered among the top methods for feature extraction or classification. In this paper, we will review two recent approaches to FLDA, namely, least squares Fisher discriminant analysis (LSFDA) and regularized kernel FDA (RKFDA) and propose deep FDA (DFDA), a straightforward nonlinear extension of LSFDA that takes advantage of the recent advances on deep neural networks. We will compare the performance of RKFDA and DFDA on a large number of two-class and multiclass problems, many of them involving class-imbalanced data sets and some having quite large sample sizes; we will use, for this, the areas under the receiver operating characteristics (ROCs) curve of the classifiers considered. As we shall see, the classification performance of both methods is often very similar and particularly good on imbalanced problems, but building DFDA models is considerably much faster than doing so for RKFDA, particularly in problems with quite large sample sizes.</abstract><cop>United States</cop><pub>IEEE</pub><pmid>30990447</pmid><doi>10.1109/TNNLS.2019.2906302</doi><tpages>12</tpages><orcidid>https://orcid.org/0000-0002-5271-0616</orcidid><orcidid>https://orcid.org/0000-0002-4002-5312</orcidid><oa>free_for_read</oa></addata></record> |
fulltext | fulltext |
identifier | ISSN: 2162-237X |
ispartof | IEEE transaction on neural networks and learning systems, 2020-08, Vol.31 (8), p.2752-2763 |
issn | 2162-237X 2162-2388 |
language | eng |
recordid | cdi_proquest_miscellaneous_2210272674 |
source | IEEE Xplore (Online service) |
subjects | Artificial neural networks Classification Covariance matrices Deep neural networks (DNNs) Discriminant analysis Encoding Feature extraction Fisher discriminant analysis (FDA) Kernel kernel discriminant analysis Least squares Linear discriminant analysis Neural networks nonlinear classifiers Proposals Training |
title | Deep Least Squares Fisher Discriminant Analysis |
url | http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-24T17%3A30%3A52IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_ieee_&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Deep%20Least%20Squares%20Fisher%20Discriminant%20Analysis&rft.jtitle=IEEE%20transaction%20on%20neural%20networks%20and%20learning%20systems&rft.au=Diaz-Vico,%20David&rft.date=2020-08-01&rft.volume=31&rft.issue=8&rft.spage=2752&rft.epage=2763&rft.pages=2752-2763&rft.issn=2162-237X&rft.eissn=2162-2388&rft.coden=ITNNAL&rft_id=info:doi/10.1109/TNNLS.2019.2906302&rft_dat=%3Cproquest_ieee_%3E2210272674%3C/proquest_ieee_%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-c395t-11d1abdd9a911bea89a2312e71dacfd3ee6ded908bc34e28d94c0ef1f952dd793%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_pqid=2431702122&rft_id=info:pmid/30990447&rft_ieee_id=8688630&rfr_iscdi=true |