End-to-End Attention-based Image Captioning

In this paper, we address the problem of image captioning specifically for molecular translation where the result would be a predicted chemical notation in InChI format for a given molecular structure. Current approaches mainly follow rule-based or CNN+RNN based methodology. However, they seem to un...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2021-04
Main Authors: Sundaramoorthy, Carola, Lin Ziwen Kelvin, Sarin, Mahak, Gupta, Shubham
Format: Article
Language:English
Subjects:
Molecular structure
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In this paper, we address the problem of image captioning specifically for molecular translation where the result would be a predicted chemical notation in InChI format for a given molecular structure. Current approaches mainly follow rule-based or CNN+RNN based methodology. However, they seem to underperform on noisy images and images with small number of distinguishable features. To overcome this, we propose an end-to-end transformer model. When compared to attention-based techniques, our proposed model outperforms on molecular datasets.
ISSN:2331-8422