Conformal Prediction with Large Language Models for Multi-Choice Question Answering

As large language models continue to be widely developed, robust uncertainty quantification techniques will become crucial for their safe deployment in high-stakes scenarios. In this work, we explore how conformal prediction can be used to provide uncertainty quantification in language models for th...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2023-07
Main Authors: Kumar, Bhawesh, Lu, Charlie, Gupta, Gauri, Palepu, Anil, Bellamy, David, Raskar, Ramesh, Beam, Andrew
Format: Article
Language:English
Subjects:
Large language models
Questions
Robustness
Safety critical
Uncertainty
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:As large language models continue to be widely developed, robust uncertainty quantification techniques will become crucial for their safe deployment in high-stakes scenarios. In this work, we explore how conformal prediction can be used to provide uncertainty quantification in language models for the specific task of multiple-choice question-answering. We find that the uncertainty estimates from conformal prediction are tightly correlated with prediction accuracy. This observation can be useful for downstream applications such as selective classification and filtering out low-quality predictions. We also investigate the exchangeability assumption required by conformal prediction to out-of-subject questions, which may be a more realistic scenario for many practical applications. Our work contributes towards more trustworthy and reliable usage of large language models in safety-critical situations, where robust guarantees of error rate are required.
ISSN:2331-8422