Retrieval-Generation Synergy Augmented Large Language Models

Large language models augmented with task-relevant documents have demonstrated impressive performance on knowledge-intensive tasks. However, regarding how to obtain effective documents, the existing methods are mainly divided into two categories. One is to retrieve from an external knowledge base, a...

Full description

Saved in:
Bibliographic Details
Main Authors: Feng, Zhangyin, Feng, Xiaocheng, Zhao, Dezhi, Yang, Maojin, Qin, Bing
Format: Conference Proceeding
Language:English
Subjects:
Acoustics
Cognition
Collaboration
Iterative methods
Knowledge based systems
large language models
question answering
Question answering (information retrieval)
retrieval augmented
Signal processing
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Large language models augmented with task-relevant documents have demonstrated impressive performance on knowledge-intensive tasks. However, regarding how to obtain effective documents, the existing methods are mainly divided into two categories. One is to retrieve from an external knowledge base, and the other is to utilize large language models to generate documents. We propose an iterative retrieval-generation collaborative framework. It is not only able to leverage both parametric and non-parametric knowledge, but also helps to find the correct reasoning path through retrieval-generation interactions, which is very important for tasks that require multi-step reasoning. We conduct experiments on four question answering datasets, including single-hop QA and multi-hop QA tasks. Empirical results show that our method significantly improves the reasoning ability of large language models and outperforms previous baselines.
ISSN:2379-190X
DOI:10.1109/ICASSP48485.2024.10448015