A parallel-fusion RNN-LSTM architecture for image caption generation

The models based on deep convolutional networks and recurrent neural networks have dominated in recent image caption generation tasks. Performance and complexity are still eternal topic. Inspired by recent work, by combining the advantages of simple RNN and LSTM, we present a novel parallel-fusion R...

Full description

Saved in:
Bibliographic Details
Main Authors: Minsi Wang, Li Song, Xiaokang Yang, Chuanfei Luo
Format: Conference Proceeding
Language:English
Subjects:
Computational modeling
Data models
deep neural network
Feature extraction
Image captioning
LSTM
Measurement
Recurrent neural networks
RNN
Training
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The models based on deep convolutional networks and recurrent neural networks have dominated in recent image caption generation tasks. Performance and complexity are still eternal topic. Inspired by recent work, by combining the advantages of simple RNN and LSTM, we present a novel parallel-fusion RNN-LSTM architecture, which obtains better results than a dominated one and improves the efficiency as well. The proposed approach divides the hidden units of RNN into several same-size parts, and lets them work in parallel. Then, we merge their outputs with corresponding ratios to generate final results. Moreover, these units can be different types of RNNs, for instance, a simple RNN and a LSTM. By training normally using NeuralTalk 1 platform on Flickr8k dataset, without additional training data, we get better results than that of dominated structure and particularly, the proposed model surpass GoogleNIC in image caption generation.
ISSN:2381-8549
DOI:10.1109/ICIP.2016.7533201