Automated reliability assessment for spectroscopic redshift measurements

• Published in: Astronomy and astrophysics (Berlin) 2018-03, Vol.611, p.A53
• Main Authors: Jamal, S., Le Brun, V., Le Fèvre, O., Vibert, D., Schmitt, A., Surace, C., Copin, Y., Garilli, B., Moresco, M., Pozzetti, L.
• Format: Article
• Language: English
• Subjects: Accuracy; Artificial intelligence; Astrophysics; Automation; Bayesian analysis; Classification; Clusters; Computer simulation; Conditional probability; Data processing; Flags; Galaxies; galaxies: distances and redshifts; Instrumentation and Methods for Astrophysic; Machine learning; methods: data analysis; methods: statistical; Polls & surveys; Probabilistic methods; Probability density functions; Quality assessment; Red shift; Reliability analysis; Sciences of the Universe; Spectroscopic analysis; surveys; techniques: spectroscopic; Test procedures
• ISSN: 0004-6361; 1432-0746; 1432-0756
• DOI: 10.1051/0004-6361/201731305

Context. Future large-scale surveys, such as the ESA Euclid mission, will produce a large set of galaxy redshifts (≥106) that will require fully automated data-processing pipelines to analyze the data, extract crucial information and ensure that all requirements are met. A fundamental element in these pipelines is to associate to each galaxy redshift measurement a quality, or reliability, estimate. Aim. In this work, we introduce a new approach to automate the spectroscopic redshift reliability assessment based on machine learning (ML) and characteristics of the redshift probability density function. Methods. We propose to rephrase the spectroscopic redshift estimation into a Bayesian framework, in order to incorporate all sources of information and uncertainties related to the redshift estimation process and produce a redshift posterior probability density function (PDF). To automate the assessment of a reliability flag, we exploit key features in the redshift posterior PDF and machine learning algorithms. Results. As a working example, public data from the VIMOS VLT Deep Survey is exploited to present and test this new methodology. We first tried to reproduce the existing reliability flags using supervised classification in order to describe different types of redshift PDFs, but due to the subjective definition of these flags (classification accuracy ~58%), we soon opted for a new homogeneous partitioning of the data into distinct clusters via unsupervised classification. After assessing the accuracy of the new clusters via resubstitution and test predictions (classification accuracy ~98%), we projected unlabeled data from preliminary mock simulations for the Euclid space mission into this mapping to predict their redshift reliability labels. Conclusions. Through the development of a methodology in which a system can build its own experience to assess the quality of a parameter, we are able to set a preliminary basis of an automated reliability assessment for spectroscopic redshift measurements. This newly-defined method is very promising for next-generation large spectroscopic surveys from the ground and in space, such as Euclid and WFIRST.