research
∙
08/25/2022
Accelerated Sparse Recovery via Gradient Descent with Nonlinear Conjugate Gradient Momentum
This paper applies an idea of adaptive momentum for the nonlinear conjug...
share
research
∙
08/11/2022
Adaptive and Implicit Regularization for Matrix Completion
The explicit low-rank regularization, e.g., nuclear norm regularization,...
research
∙
08/01/2022
Momentum Transformer: Closing the Performance Gap Between Self-attention and Its Linearization
Transformers have achieved remarkable success in sequence modeling and b...
research
∙
04/19/2022
Proximal Implicit ODE Solvers for Accelerating Learning Neural ODEs
Learning neural ODEs often requires solving very stiff ODE systems, prim...
research
∙
02/24/2022
Learning POD of Complex Dynamics Using Heavy-ball Neural ODEs
Proper orthogonal decomposition (POD) allows reduced-order modeling of c...
research
∙
01/22/2022
glassoformer: a query-sparse transformer for post-fault power grid voltage prediction
We propose GLassoformer, a novel and efficient transformer architecture ...
research
∙
12/12/2021
Efficient and Reliable Overlay Networks for Decentralized Federated Learning
We propose near-optimal overlay networks based on d-regular expander gra...
research
∙
10/18/2021
Training Deep Neural Networks with Adaptive Momentum Inspired by the Quadratic Optimization
Heavy ball momentum is crucial in accelerating (stochastic) gradient-bas...
research
∙
10/13/2021
How Does Momentum Benefit Deep Neural Networks Architecture Design? A Few Case Studies
We present and review an algorithmic and theoretical framework for impro...
research
∙
10/10/2021
Heavy Ball Neural Ordinary Differential Equations
We propose heavy ball neural ordinary differential equations (HBNODEs), ...
research
∙
08/05/2021
FMMformer: Efficient and Flexible Transformer via Decomposed Near-field and Far-field Attention
We propose FMMformers, a class of efficient and flexible transformers in...
research
∙
04/23/2021
Decentralized Federated Averaging
Federated averaging (FedAvg) is a communication efficient algorithm for ...
research
∙
04/22/2021
Robust Certification for Laplace Learning on Geometric Graphs
Graph Laplacian (GL)-based semi-supervised learning is one of the most u...
research
∙
02/02/2021
Stability and Generalization of the Decentralized Stochastic Gradient Descent
The stability and generalization of stochastic gradient-based methods pr...
research
∙
12/03/2020
Stochastic Gradient Descent with Nonlinear Conjugate Gradient-Style Adaptive Momentum
Momentum plays a crucial role in stochastic gradient-based optimization ...
research
∙
11/30/2020
Deep Interactive Denoiser (DID) for X-Ray Computed Tomography
Low dose computed tomography (LDCT) is desirable for both diagnostic ima...
research
∙
08/31/2020
An Integrated Approach to Produce Robust Models with High Efficiency
Deep Neural Networks (DNNs) needs to be both efficient and robust for pr...
research
∙
06/12/2020
MomentumRNN: Integrating Momentum into Recurrent Neural Networks
Designing deep neural networks is an art that often involves an expensiv...
research
∙
05/01/2020
Exploring Private Federated Learning with Laplacian Smoothing
Federated learning aims to protect data privacy by collaboratively learn...
research
∙
03/02/2020
Sparsity Meets Robustness: Channel Pruning for the Feynman-Kac Formalism Principled Robust Deep Neural Nets
Deep neural nets (DNNs) compression is crucial for adaptation to mobile ...
research
∙
02/24/2020
Scheduled Restart Momentum for Accelerated Stochastic Gradient Descent
Stochastic gradient descent (SGD) with constant momentum and its variant...
research
∙
11/02/2019
Laplacian Smoothing Stochastic Gradient Markov Chain Monte Carlo
As an important Markov Chain Monte Carlo (MCMC) method, stochastic gradi...
research
∙
07/16/2019
Graph Interpolating Activation Improves Both Natural and Robust Accuracies in Data-Efficient Deep Learning
Improving the accuracy and robustness of deep neural nets (DNNs) and ada...
research
∙
06/28/2019
DP-LSSGD: A Stochastic Optimization Method to Lift the Utility in Privacy-Preserving ERM
Machine learning (ML) models trained by differentially private stochasti...
research
∙
02/13/2019
A Study on Graph-Structured Recurrent Neural Networks and Sparsification with Application to Epidemic Forecasting
We study epidemic forecasting on real-world health data by a graph-struc...
research
∙
01/21/2019
A Deterministic Approach to Avoid Saddle Points
Loss functions with a large number of saddle points are one of the main ...
research
∙
11/26/2018
EnResNet: ResNet Ensemble via the Feynman-Kac Formalism
We propose a simple yet powerful ResNet ensemble algorithm which consist...
research
∙
11/15/2018
Mathematical Analysis of Adversarial Attacks
In this paper, we analyze efficacy of the fast gradient sign method (FGS...
research
∙
09/23/2018
Adversarial Defense via Data Dependent Activation Function and Total Variation Minimization
We improve the robustness of deep neural nets to adversarial attacks by ...
research
∙
06/17/2018
Laplacian Smoothing Gradient Descent
We propose a very simple modification of gradient descent and stochastic...
research
∙
05/18/2018
Stop memorizing: A data-dependent regularization framework for intrinsic pattern learning
Deep neural networks (DNNs) typically have enough capacity to fit random...
research
∙
04/02/2018
Graph-Based Deep Modeling and Real Time Forecasting of Sparse Spatio-Temporal Data
We present a generic framework for spatio-temporal (ST) data modeling, a...
research
∙
02/01/2018
Deep Learning with Data Dependent Implicit Activation Function
Though deep neural networks (DNNs) achieve remarkable performances in ma...
research
∙
11/23/2017
Deep Learning for Real-Time Crime Forecasting and its Ternarization
Real-time crime forecasting is important. However, accurate prediction o...
research
∙
07/09/2017
