Zheng Zhan

I am a fourth-year PhD student in Machine Learning at Northeastern University, under the supervision of Prof. Yanzhi Wang. I also work closely with Prof. Stratis Ioannidis, and Prof. Jennifer G. Dy.

Email  /  Google Scholar  /  LinkedIn  /  Resume

• Model compression
• Continual (Lifelong) learning
• Image super-resolution
• Deep learning applications on resource-limited platforms/scenarios.

• 02/2024: Our paper LOTUS is accepted at DAC 2024
• 08/2023: Our paper MOC is accepted at ICCAD 2023
• 04/2023: Our paper DualHISC is accepted at ICML 2023
• 02/2023: Our paper Condense is accepted at DAC 2023
• 09/2022: Our paper on efficient continual learning is accepted at NeurIPS 2022
• 07/2022: Our paper All-in-One is accepted at ICCAD 2022
• 07/2022: Our paper on compiler-aware NAS for real-time super-resolution on mobile is accepted at ECCV 2022
• 11/2021: Our paper on automatic pruning scheme mapping is accepted at TODAES
• 09/2021: Our paper on memory-Economic sparse training is accepted at NeurIPS 2021
• 07/2021: Our paper on NAS and pruning search for real-time super-resolution on mobile is accepted at ICCV 2021
• 02/2021: Our paper NPAS is accepted at CVPR 2021
• 02/2021: Our paper on weight pruning using reweighted optimization methods is accepted at DAC 2021
• Earlier news
06/2020: Our paper on privacy-preserving-oriented DNN pruning is invited at GLSVLSI 2021
04/2019: Our paper Chic is accepted at IWQoS 2019
11/2018: Our paper on universal approximation property and equivalence is accepted at AAAI 2019

DualHSIC presents a method to improve rehearsal based approach for continual learning. The basic idea proposed here is to leverage inter-task relationships using two concepts related to the Hilbert Schmidt independence criterion (HSIC). One component (HSIC-Bottleneck for Rehearsal (HBR)) helps reduce interference between tasks and the other component (HSIC Alignment - HA) helps share task-invariant knowledge.

Propose a two-level algorithm for obtaining subnets with arbitrary ratios in a single model with theoretical proof. Develop a framework which leverages the DVFS and compression techniques to get multiple subnetworks in one neural network to lower the variance of inference runtime for different hardware frequency levels. It’s a much more automatic framework.

SparCL explores sparsity for efficient continual learning and achieves both training acceleration and accuracy preservation through the synergy of three aspects: weight sparsity, data efficiency, and gradient sparsity.

All-in-One, a highly representative pruning framework to work with dynamic power management using DVFS. The framework can use only one set of model weights and soft masks to represent multiple models of various pruning ratios. By re-configuring the model to the corresponding pruning ratio for a specific execution frequency (and voltage), we can keep the difference in speed performance under various execution frequencies as small as possible.

We propose a compiler-aware SR neural architecture search (NAS) framework that conducts depth search and per-layer width search with adaptive SR blocks. A speed model incorporated with compiler optimizations is leveraged to predict the inference latency of the SR block with various width configurations for faster convergence.

We proposes to use neural architecture search and network pruning to find a highly efficient network for image super-resolution. we are the first to achieve real-time SR inference (with only tens of milliseconds per frame) for implementing 720p resolution with competitive perceptual performance on mobile platforms.

We propose a unified DNN weight pruning framework with dynamically updated regularization terms bounded by the designated constraint, which can generate both non-structured sparsity and different kinds of structured sparsity.

We prove that the ”ideal” SCNNs and BNNs satisfy the universal approximation property with probability 1 (due to the stochastic behavior), we further prove that SCNNs and BNNs exhibit the same energy complexity.