research
∙
11/24/2021
Sparse is Enough in Scaling Transformers
Large Transformer models yield impressive results on many tasks, but are...
share
research
∙
10/26/2021
Hierarchical Transformers Are More Efficient Language Models
Transformer models yield impressive results on many NLP and sequence mod...
research
∙
07/07/2021
Evaluating Large Language Models Trained on Code
We introduce Codex, a GPT language model fine-tuned on publicly availabl...
research
∙
02/12/2021
Q-Value Weighted Regression: Reinforcement Learning with Limited Data
Sample efficiency and performance in the offline setting have emerged as...
research
∙
09/30/2020
Rethinking Attention with Performers
We introduce Performers, Transformer architectures which can estimate re...
research
∙
01/13/2020
Reformer: The Efficient Transformer
Large Transformer models routinely achieve state-of-the-art results on a...
research
∙
06/11/2019
Parallel Scheduled Sampling
Auto-regressive models are widely used in sequence generation problems. ...
research
∙
05/21/2019
Sample Efficient Text Summarization Using a Single Pre-Trained Transformer
Language model (LM) pre-training has resulted in impressive performance ...
research
∙
03/01/2019
Model-Based Reinforcement Learning for Atari
Model-free reinforcement learning (RL) can be used to learn effective po...
research
∙
10/23/2018
Area Attention
Existing attention mechanisms, are mostly item-based in that a model is ...
research
∙
07/10/2018
Universal Transformers
Self-attentive feed-forward sequence models have been shown to achieve i...
research
∙
03/16/2018
Tensor2Tensor for Neural Machine Translation
Tensor2Tensor is a library for deep learning models that is well-suited ...
research
∙
03/09/2018
Fast Decoding in Sequence Models using Discrete Latent Variables
Autoregressive sequence models based on deep neural networks, such as RN...
research
∙
02/15/2018
Image Transformer
Image generation has been successfully cast as an autoregressive sequenc...
research
∙
02/15/2018
Image Tranformer
Image generation has been successfully cast as an autoregressive sequenc...
research
∙
01/30/2018
Generating Wikipedia by Summarizing Long Sequences
We show that generating English Wikipedia articles can be approached as ...
research
∙
01/29/2018
Discrete Autoencoders for Sequence Models
Recurrent models for sequences have been recently successful at many tas...
research
∙
01/15/2018
Unsupervised Cipher Cracking Using Discrete GANs
This work details CipherGAN, an architecture inspired by CycleGAN used f...
research
∙
06/16/2017
One Model To Learn Them All
Deep learning yields great results across many fields, from speech recog...
research
∙
06/12/2017
Attention Is All You Need
The dominant sequence transduction models are based on complex recurrent...
research
∙
06/09/2017
Depthwise Separable Convolutions for Neural Machine Translation
Depthwise separable convolutions reduce the number of parameters and com...
research
∙
01/23/2017
Regularizing Neural Networks by Penalizing Confident Output Distributions
We systematically explore regularizing neural networks by penalizing low...
research
∙
10/27/2016
Can Active Memory Replace Attention?
Several mechanisms to focus attention of a neural network on selected pa...
research
∙
09/26/2016
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Neural Machine Translation (NMT) is an end-to-end learning approach for ...
research
∙
03/14/2016
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
TensorFlow is an interface for expressing machine learning algorithms, a...
research
∙
11/25/2015
Neural GPUs Learn Algorithms
Learning an algorithm from examples is a fundamental problem that has be...
research
∙
11/21/2015
Adding Gradient Noise Improves Learning for Very Deep Networks
Deep feedforward and recurrent networks have achieved impressive results...
research
∙
11/19/2015
Multi-task Sequence to Sequence Learning
Sequence to sequence learning has recently emerged as a new paradigm in ...
research
∙
12/23/2014
