Loading...
Loading...

Go to the content (press return)

Skip RNN: learning to skip state updates in recurrent neural networks

Author
Campos, V.; Jou, B.; Giro, X.; Torres, J.; Chang, S.
Type of activity
Presentation of work at congresses
Name of edition
6th International Conference on Learning Representations
Date of publication
2018
Presentation's date
2018-05-03
Book of congress proceedings
Sixth International Conference on Learning Representations: Monday April 30-Thursday May 03, 2018, Vancouver Convention Center, Vancouver: [proceedings]
First page
1
Last page
17
Project funding
High performance computing VII
Multimodal Signal Processing and Machine Learning on Graphs
Repository
http://hdl.handle.net/2117/118098 Open in new window
URL
https://iclr.cc/Conferences/2018/Schedule?type=Poster Open in new window
Abstract
Recurrent Neural Networks (RNNs) continue to show outstanding performance in sequence modeling tasks. However, training RNNs on long sequences often face challenges like slow inference, vanishing gradients and difficulty in capturing long term dependencies. In backpropagation through time settings, these issues are tightly coupled with the large, sequential computational graph resulting from unfolding the RNN in time. We introduce the Skip RNN model which extends existing RNN models by learning ...
Citation
Campos, V., Jou, B., Giro, X., Torres, J., Chang, S. Skip RNN: learning to skip state updates in recurrent neural networks. A: International Conference on Learning Representations. "Sixth International Conference on Learning Representations: Monday April 30-Thursday May 03, 2018, Vancouver Convention Center, Vancouver: [proceedings]". 2018, p. 1-17.
Keywords
conditional computation, dynamic learning, recurrent neural networks
Group of research
CAP - High Performace Computing Group
GPI - Image and Video Processing Group
IDEAI-UPC - Intelligent Data Science and Artificial Intelligence Research Center

Participants

  • Campos Camúñez, Victor  (author and speaker )
  • Jou, Brendan  (author and speaker )
  • Giro Nieto, Xavier  (author and speaker )
  • Torres Viñals, Jordi  (author and speaker )
  • Chang, Shih-Fu  (author and speaker )

Attachments