Reformer and Terraformer are more efficient versions of Transformer that uses reversible layers, locality-sensitive hashing and sparse layers.
-
Reformer: Read about the details of Reformer in the Reformer paper which was selected for oral presentation at ICLR 2020.
-
Terraformer: Read about the details of Terraformer in the following paper.
-
Generate images with Reformer using this colab.
-
Translate from English to German with a reversible encoder-decoder model using this colab.
-
Medium-size (~700M weights) Terraformer model for summarizing arxiv articles is available at
gs://trax-ml/terraformer/medium -
Large (~7B weights) Terraformer model pre-trained on C4
gs://trax-ml/terraformer/big