Contrastive Training Improves Zero-Shot Classification of Semi-structured Documents

We investigate semi-structured document classification in a zero-shot setting. Classification of semi-structured documents is more challenging than that of standard unstructured documents, as positional, layout, and style information play a vital role in interpreting such documents. The standard cla...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2022-10
Main Authors: Khalifa, Muhammad, Vyas, Yogarshi, Wang, Shuai, Horwood, Graham, Mallya, Sunil, Ballesteros, Miguel
Format: Article
Language:English
Subjects:
Classification
Documents
Training
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We investigate semi-structured document classification in a zero-shot setting. Classification of semi-structured documents is more challenging than that of standard unstructured documents, as positional, layout, and style information play a vital role in interpreting such documents. The standard classification setting where categories are fixed during both training and testing falls short in dynamic environments where new document categories could potentially emerge. We focus exclusively on the zero-shot setting where inference is done on new unseen classes. To address this task, we propose a matching-based approach that relies on a pairwise contrastive objective for both pretraining and fine-tuning. Our results show a significant boost in Macro F\(_1\) from the proposed pretraining step in both supervised and unsupervised zero-shot settings.
ISSN:2331-8422