Outlier Prediction and Training Set Modification to Reduce Catastrophic Outlier Redshift Estimates in Large-scale Surveys

We present results of using individual galaxies’ probability distribution over redshift as a method of identifying potential catastrophic outliers in empirical photometric redshift estimation. In the course of developing this approach we develop a method of modification of the redshift distribution...

Full description

Saved in:
Bibliographic Details
Published in:Publications of the Astronomical Society of the Pacific 2021-04, Vol.133 (1022), p.44504
Main Authors: Wyatt, M., Singal, J.
Format: Article
Language:English
Subjects:
Galaxies
Mitigation
Optimization
Polls & surveys
Probability distribution
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by cdi_FETCH-LOGICAL-c313t-dacf10a050ce9f5da7e1ec49c53c5591109ab71c013bca9e9b0eef5f789f87a83
cites cdi_FETCH-LOGICAL-c313t-dacf10a050ce9f5da7e1ec49c53c5591109ab71c013bca9e9b0eef5f789f87a83
container_end_page
container_issue 1022
container_start_page 44504
container_title Publications of the Astronomical Society of the Pacific
container_volume 133
creator Wyatt, M.
Singal, J.
description We present results of using individual galaxies’ probability distribution over redshift as a method of identifying potential catastrophic outliers in empirical photometric redshift estimation. In the course of developing this approach we develop a method of modification of the redshift distribution of training sets to improve both the baseline accuracy of high redshift ( z > 1.5) estimation as well as catastrophic outlier mitigation. We demonstrate these using two real test data sets and one simulated test data set spanning a wide redshift range (0 < z < 4). Results presented here inform an example “prescription” that can be applied as a realistic photometric redshift estimation scenario for a hypothetical large-scale survey. We find that with appropriate optimization, we can identify a significant percentage (>30%) of catastrophic outlier galaxies while simultaneously incorrectly flagging only a small percentage ( 1.5 with only a small (
doi_str_mv 10.1088/1538-3873/abe5fb
format article
fullrecord <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_journals_2511960534</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2511960534</sourcerecordid><originalsourceid>FETCH-LOGICAL-c313t-dacf10a050ce9f5da7e1ec49c53c5591109ab71c013bca9e9b0eef5f789f87a83</originalsourceid><addsrcrecordid>eNo9kM1PAjEQxRujifhx99jE88oMpWx7NAQ_EgxG8Nx0u1MowV1suyb894Kop5fMe3nz8mPsBuEOQak-SqEKoUrRtxVJX52w3v_plPUAYFiMBgrO2UVKawBEhdBju1mXN4Eif41UB5dD23Db1HwRbWhCs-RzyvylrYMPzv64ueVvVHeO-Nhmm3Jst6vg-F_P3kur4DOfpBw-bKbEQ8OnNi6pSM5uiM-7-EW7dMXOvN0kuv7VS_b-MFmMn4rp7PF5fD8tnECRi9o6j2BBgiPtZW1LQnJD7aRwUmpE0LYq0QGKyllNugIiL32ptFelVeKS3R57t7H97Chls2672OxfmoFE1COQYrhPwTHlYptSJG-2cT8_7gyCOQA2B5rmQNMcAYtvbm1xsg</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2511960534</pqid></control><display><type>article</type><title>Outlier Prediction and Training Set Modification to Reduce Catastrophic Outlier Redshift Estimates in Large-scale Surveys</title><source>JSTOR Archival Journals and Primary Sources Collection</source><source>Institute of Physics:Jisc Collections:IOP Publishing Read and Publish 2024-2025 (Reading List)</source><creator>Wyatt, M. ; Singal, J.</creator><creatorcontrib>Wyatt, M. ; Singal, J.</creatorcontrib><description><![CDATA[We present results of using individual galaxies’ probability distribution over redshift as a method of identifying potential catastrophic outliers in empirical photometric redshift estimation. In the course of developing this approach we develop a method of modification of the redshift distribution of training sets to improve both the baseline accuracy of high redshift ( z > 1.5) estimation as well as catastrophic outlier mitigation. We demonstrate these using two real test data sets and one simulated test data set spanning a wide redshift range (0 < z < 4). Results presented here inform an example “prescription” that can be applied as a realistic photometric redshift estimation scenario for a hypothetical large-scale survey. We find that with appropriate optimization, we can identify a significant percentage (>30%) of catastrophic outlier galaxies while simultaneously incorrectly flagging only a small percentage (<7% and in many cases <3%) of non-outlier galaxies as catastrophic outliers. We find also that our training set redshift distribution modification results in a significant (>10) percentage point decrease of outlier galaxies for z > 1.5 with only a small (<3) percentage point increase of outlier galaxies for z < 1.5 compared to the unmodified training set. In addition, we find that this modification can in some cases cause a significant (∼20) percentage point decrease of galaxies which are non-outliers but which have been incorrectly identified as outliers, while in other cases cause only a small (<1) increase in this metric.]]></description><identifier>ISSN: 0004-6280</identifier><identifier>EISSN: 1538-3873</identifier><identifier>DOI: 10.1088/1538-3873/abe5fb</identifier><language>eng</language><publisher>Philadelphia: IOP Publishing</publisher><subject>Galaxies ; Mitigation ; Optimization ; Polls &amp; surveys ; Probability distribution</subject><ispartof>Publications of the Astronomical Society of the Pacific, 2021-04, Vol.133 (1022), p.44504</ispartof><rights>Copyright IOP Publishing Apr 2021</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c313t-dacf10a050ce9f5da7e1ec49c53c5591109ab71c013bca9e9b0eef5f789f87a83</citedby><cites>FETCH-LOGICAL-c313t-dacf10a050ce9f5da7e1ec49c53c5591109ab71c013bca9e9b0eef5f789f87a83</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>314,776,780,27901,27902</link.rule.ids></links><search><creatorcontrib>Wyatt, M.</creatorcontrib><creatorcontrib>Singal, J.</creatorcontrib><title>Outlier Prediction and Training Set Modification to Reduce Catastrophic Outlier Redshift Estimates in Large-scale Surveys</title><title>Publications of the Astronomical Society of the Pacific</title><description><![CDATA[We present results of using individual galaxies’ probability distribution over redshift as a method of identifying potential catastrophic outliers in empirical photometric redshift estimation. In the course of developing this approach we develop a method of modification of the redshift distribution of training sets to improve both the baseline accuracy of high redshift ( z > 1.5) estimation as well as catastrophic outlier mitigation. We demonstrate these using two real test data sets and one simulated test data set spanning a wide redshift range (0 < z < 4). Results presented here inform an example “prescription” that can be applied as a realistic photometric redshift estimation scenario for a hypothetical large-scale survey. We find that with appropriate optimization, we can identify a significant percentage (>30%) of catastrophic outlier galaxies while simultaneously incorrectly flagging only a small percentage (<7% and in many cases <3%) of non-outlier galaxies as catastrophic outliers. We find also that our training set redshift distribution modification results in a significant (>10) percentage point decrease of outlier galaxies for z > 1.5 with only a small (<3) percentage point increase of outlier galaxies for z < 1.5 compared to the unmodified training set. In addition, we find that this modification can in some cases cause a significant (∼20) percentage point decrease of galaxies which are non-outliers but which have been incorrectly identified as outliers, while in other cases cause only a small (<1) increase in this metric.]]></description><subject>Galaxies</subject><subject>Mitigation</subject><subject>Optimization</subject><subject>Polls &amp; surveys</subject><subject>Probability distribution</subject><issn>0004-6280</issn><issn>1538-3873</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2021</creationdate><recordtype>article</recordtype><recordid>eNo9kM1PAjEQxRujifhx99jE88oMpWx7NAQ_EgxG8Nx0u1MowV1suyb894Kop5fMe3nz8mPsBuEOQak-SqEKoUrRtxVJX52w3v_plPUAYFiMBgrO2UVKawBEhdBju1mXN4Eif41UB5dD23Db1HwRbWhCs-RzyvylrYMPzv64ueVvVHeO-Nhmm3Jst6vg-F_P3kur4DOfpBw-bKbEQ8OnNi6pSM5uiM-7-EW7dMXOvN0kuv7VS_b-MFmMn4rp7PF5fD8tnECRi9o6j2BBgiPtZW1LQnJD7aRwUmpE0LYq0QGKyllNugIiL32ptFelVeKS3R57t7H97Chls2672OxfmoFE1COQYrhPwTHlYptSJG-2cT8_7gyCOQA2B5rmQNMcAYtvbm1xsg</recordid><startdate>20210401</startdate><enddate>20210401</enddate><creator>Wyatt, M.</creator><creator>Singal, J.</creator><general>IOP Publishing</general><scope>AAYXX</scope><scope>CITATION</scope><scope>7TG</scope><scope>KL.</scope></search><sort><creationdate>20210401</creationdate><title>Outlier Prediction and Training Set Modification to Reduce Catastrophic Outlier Redshift Estimates in Large-scale Surveys</title><author>Wyatt, M. ; Singal, J.</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c313t-dacf10a050ce9f5da7e1ec49c53c5591109ab71c013bca9e9b0eef5f789f87a83</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2021</creationdate><topic>Galaxies</topic><topic>Mitigation</topic><topic>Optimization</topic><topic>Polls &amp; surveys</topic><topic>Probability distribution</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Wyatt, M.</creatorcontrib><creatorcontrib>Singal, J.</creatorcontrib><collection>CrossRef</collection><collection>Meteorological &amp; Geoastrophysical Abstracts</collection><collection>Meteorological &amp; Geoastrophysical Abstracts - Academic</collection><jtitle>Publications of the Astronomical Society of the Pacific</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Wyatt, M.</au><au>Singal, J.</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Outlier Prediction and Training Set Modification to Reduce Catastrophic Outlier Redshift Estimates in Large-scale Surveys</atitle><jtitle>Publications of the Astronomical Society of the Pacific</jtitle><date>2021-04-01</date><risdate>2021</risdate><volume>133</volume><issue>1022</issue><spage>44504</spage><pages>44504-</pages><issn>0004-6280</issn><eissn>1538-3873</eissn><abstract><![CDATA[We present results of using individual galaxies’ probability distribution over redshift as a method of identifying potential catastrophic outliers in empirical photometric redshift estimation. In the course of developing this approach we develop a method of modification of the redshift distribution of training sets to improve both the baseline accuracy of high redshift ( z > 1.5) estimation as well as catastrophic outlier mitigation. We demonstrate these using two real test data sets and one simulated test data set spanning a wide redshift range (0 < z < 4). Results presented here inform an example “prescription” that can be applied as a realistic photometric redshift estimation scenario for a hypothetical large-scale survey. We find that with appropriate optimization, we can identify a significant percentage (>30%) of catastrophic outlier galaxies while simultaneously incorrectly flagging only a small percentage (<7% and in many cases <3%) of non-outlier galaxies as catastrophic outliers. We find also that our training set redshift distribution modification results in a significant (>10) percentage point decrease of outlier galaxies for z > 1.5 with only a small (<3) percentage point increase of outlier galaxies for z < 1.5 compared to the unmodified training set. In addition, we find that this modification can in some cases cause a significant (∼20) percentage point decrease of galaxies which are non-outliers but which have been incorrectly identified as outliers, while in other cases cause only a small (<1) increase in this metric.]]></abstract><cop>Philadelphia</cop><pub>IOP Publishing</pub><doi>10.1088/1538-3873/abe5fb</doi><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier ISSN: 0004-6280
ispartof Publications of the Astronomical Society of the Pacific, 2021-04, Vol.133 (1022), p.44504
issn 0004-6280
1538-3873
language eng
recordid cdi_proquest_journals_2511960534
source JSTOR Archival Journals and Primary Sources Collection; Institute of Physics:Jisc Collections:IOP Publishing Read and Publish 2024-2025 (Reading List)
subjects Galaxies
Mitigation
Optimization
Polls & surveys
Probability distribution
title Outlier Prediction and Training Set Modification to Reduce Catastrophic Outlier Redshift Estimates in Large-scale Surveys
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-30T22%3A26%3A34IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Outlier%20Prediction%20and%20Training%20Set%20Modification%20to%20Reduce%20Catastrophic%20Outlier%20Redshift%20Estimates%20in%20Large-scale%20Surveys&rft.jtitle=Publications%20of%20the%20Astronomical%20Society%20of%20the%20Pacific&rft.au=Wyatt,%20M.&rft.date=2021-04-01&rft.volume=133&rft.issue=1022&rft.spage=44504&rft.pages=44504-&rft.issn=0004-6280&rft.eissn=1538-3873&rft_id=info:doi/10.1088/1538-3873/abe5fb&rft_dat=%3Cproquest_cross%3E2511960534%3C/proquest_cross%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-c313t-dacf10a050ce9f5da7e1ec49c53c5591109ab71c013bca9e9b0eef5f789f87a83%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_pqid=2511960534&rft_id=info:pmid/&rfr_iscdi=true