Hierarchically-Attentive RNN for Album Summarization and Storytelling

Licheng Yu, Mohit Bansal, Tamara Berg


Abstract
We address the problem of end-to-end visual storytelling. Given a photo album, our model first selects the most representative (summary) photos, and then composes a natural language story for the album. For this task, we make use of the Visual Storytelling dataset and a model composed of three hierarchically-attentive Recurrent Neural Nets (RNNs) to: encode the album photos, select representative (summary) photos, and compose the story. Automatic and human evaluations show our model achieves better performance on selection, generation, and retrieval than baselines.
Anthology ID:
D17-1101
Volume:
Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing
Month:
September
Year:
2017
Address:
Copenhagen, Denmark
Editors:
Martha Palmer, Rebecca Hwa, Sebastian Riedel
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
966–971
Language:
URL:
https://aclanthology.org/D17-1101
DOI:
10.18653/v1/D17-1101
Bibkey:
Cite (ACL):
Licheng Yu, Mohit Bansal, and Tamara Berg. 2017. Hierarchically-Attentive RNN for Album Summarization and Storytelling. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 966–971, Copenhagen, Denmark. Association for Computational Linguistics.
Cite (Informal):
Hierarchically-Attentive RNN for Album Summarization and Storytelling (Yu et al., EMNLP 2017)
Copy Citation:
PDF:
https://aclanthology.org/D17-1101.pdf
Attachment:
 D17-1101.Attachment.pdf
Data
VIST