Echo State Networks are easy-to-train recurrent neural networks, a variant of Reservoir Computing. In some sense, these networks show how far you can get with nothing but a good weight initialisation.

This ESN implementation is relatively simple and self-contained, though it offers tricks like noise injection and teacher forcing (feedback connections), plus a zoo of dubious little hyperparameters.

However! If your aims are practical and your gradients automatic, consider using a fully trained network.

• learning to be a tunable frequency generator
• learning a Mackey-Glass system