Scribd
Upload
22 views22 pages

Lecture 11

The lecture covers sequence models, focusing on recurrent neural networks, the vanishing and exploding gradients problem, and long-short term memory (LSTM) networks. It discusses LSTM applications including language models, translation, caption generation, and program execution. Additionally, it touches on advanced topics like Neural Turing Machines and various prediction and recognition tasks.

Uploaded by

Tachbir Dewan
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
22 views22 pages

Lecture 11

The lecture covers sequence models, focusing on recurrent neural networks, the vanishing and exploding gradients problem, and long-short term memory (LSTM) networks. It discusses LSTM applications including language models, translation, caption generation, and program execution. Additionally, it touches on advanced topics like Neural Turing Machines and various prediction and recognition tasks.

Uploaded by

Tachbir Dewan
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 22

Outline of the lecture

This lecture introduces you sequence models. The goal is for you to
learn about:

 Recurrent neural networks


 The vanishing and exploding gradients problem
 Long-short term memory (LSTM) networks
 Applications of LSTM networks
 Language models
 Translation
 Caption generation
 Program execution
A simple recurrent neural network

[Alex Graves]
Vanishing gradient problem

[Yoshua Bengio et al]


Vanishing gradient problem
Simple solution
LSTM

[Alex Graves]
LSTM
Entry-wise multiplication layer
LSTM cell in Torch
LSTM column in Torch
LSTMs for sequence to sequence prediction

[Ilya Sutskever et al]


LSTMs for sequence to sequence prediction
Learning to parse

[Oriol Vinyals et al]


Learning to execute

[Wojciech Zaremba and Ilya Sutskever]


Video prediction
Hand-writing recognition and synthesis

[Alex Graves]
Neural Turing Machine (NTM)

[Alex Graves, Greg Wayne, Ivo Danihelka]


Neural Turing Machine (NTM)
Neural Turing Machine (NTM)
Translation with alignment (Bahdanau et al)
Show, attend and tell

[Kelvin Xu et al, 2015]


Show, attend and tell

You might also like