research
∙
08/13/2023
Token-Scaled Logit Distillation for Ternary Weight Generative Language Models
Generative Language Models (GLMs) have shown impressive performance in t...
share
research
∙
02/23/2023
Teacher Intervention: Improving Convergence of Quantization Aware Training for Ultra-Low Precision Transformers
Pre-trained Transformer models such as BERT have shown great success in ...
research
∙
01/29/2023
Exploring Attention Map Reuse for Efficient Transformer Neural Networks
Transformer-based deep neural networks have achieved great success in va...
research
∙
12/29/2022
Macro-block dropout for improved regularization in training end-to-end speech recognition models
This paper proposes a new regularization algorithm referred to as macro-...
research
∙
10/01/2022
A Comparison of Transformer, Convolutional, and Recurrent Neural Networks on Phoneme Recognition
Phoneme recognition is a very important part of speech recognition that ...
research
∙
03/19/2022
Similarity and Content-based Phonetic Self Attention for Speech Recognition
Transformer-based speech recognition models have achieved great success ...
research
∙
02/22/2022
Korean Tokenization for Beam Search Rescoring in Speech Recognition
The performance of automatic speech recognition (ASR) models can be grea...
research
∙
10/07/2021
Layer-wise Pruning of Transformer Attention Heads for Efficient Language Modeling
While Transformer-based models have shown impressive language modeling p...
research
∙
09/30/2020
Stochastic Precision Ensemble: Self-Knowledge Distillation for Quantized Deep Neural Networks
The quantization of deep neural networks (QDNNs) has been actively studi...
research
∙
09/05/2020
S-SGD: Symmetrical Stochastic Gradient Descent with Weight Noise Injection for Reaching Flat Minima
The stochastic gradient descent (SGD) method is most widely used for dee...
research
∙
05/31/2020
Quantized Neural Networks: Characterization and Holistic Optimization
Quantized deep neural networks (QDNNs) are necessary for low-power, high...
research
∙
02/02/2020
SQWA: Stochastic Quantized Weight Averaging for Improving the Generalization Capability of Low-Precision Deep Neural Networks
Designing a deep neural network (DNN) with good generalization capabilit...
research
∙
09/04/2019
Empirical Analysis of Knowledge Distillation Technique for Optimization of Quantized Deep Neural Networks
Knowledge distillation (KD) is a very popular method for model size redu...
research
∙
11/05/2018
Workload-aware Automatic Parallelization for Multi-GPU DNN Training
Deep neural networks (DNNs) have emerged as successful solutions for var...
research
∙
03/30/2018
Single Stream Parallelization of Recurrent Neural Networks for Low Power and Fast Inference
As neural network algorithms show high performance in many applications,...
research
∙
07/01/2017
Structured Sparse Ternary Weight Coding of Deep Neural Networks for Efficient Hardware Implementations
Deep neural networks (DNNs) usually demand a large amount of operations ...
research
∙
11/19/2016
Quantized neural network design under weight capacity constraint
The complexity of deep neural network algorithms for hardware implementa...
research
∙
10/30/2016
Compact Deep Convolutional Neural Networks With Coarse Pruning
The learning capability of a neural network improves with increasing dep...
research
∙
09/13/2016
Character-Level Language Modeling with Hierarchical Recurrent Neural Networks
Recurrent neural network (RNN) based character-level language models (CL...
research
∙
08/14/2016
Dynamic Hand Gesture Recognition for Wearable Devices with Low Complexity Recurrent Neural Networks
Gesture recognition is a very essential technology for many wearable dev...
research
∙
02/04/2016
FPGA Based Implementation of Deep Neural Networks Using On-chip Memory Only
Deep neural networks (DNNs) demand a very large amount of computation an...
research
∙
01/25/2016
Character-Level Incremental Speech Recognition with Recurrent Neural Networks
In real-time speech recognition applications, the latency is an importan...
research
∙
12/30/2015
Online Keyword Spotting with a Character-Level Recurrent Neural Network
In this paper, we propose a context-aware keyword spotting model employi...
research
∙
12/29/2015
Structured Pruning of Deep Convolutional Neural Networks
Real time application of deep learning algorithms is often hindered by h...
research
∙
12/04/2015
Fixed-Point Performance Analysis of Recurrent Neural Networks
Recurrent neural networks have shown excellent performance in many appli...
research
∙
11/21/2015
Online Sequence Training of Recurrent Neural Networks with Connectionist Temporal Classification
Connectionist temporal classification (CTC) based supervised sequence tr...
research
∙
11/20/2015
Resiliency of Deep Neural Networks under Quantization
The complexity of deep neural network algorithms for hardware implementa...
research
∙
03/10/2015
