Modern machine learning, especially deep learning, faces a fundamental question: how to create models that efficiently deliver reliable predictions to meet the requirements of diverse applications running on various systems. This talk will introduce reuse-centric optimization, a novel direction for addressing the fundamental question. Reuse-centric optimization centers around harnessing reuse opportunities for enhancing computing efficiency. It generalizes the principle to a higher level and a larger scope through a synergy between programming systems and machine learning algorithms. Its exploitation of computation reuse spans across the boundaries of machine learning algorithms, implementations, and infrastructures; the types of reuse it covers range from pre-trained Neural Network building blocks to preprocessed results and even memory bits; the scopes of reuse it leverages go from training pipelines of deep learning to variants of Neural Networks in ensembles; the benefits it generates extend from orders of magnitude faster search for a good smaller Convolution Neural Network (CNN) to the elimination of all space cost in protecting parameters of CNNs.
Speaker Biography
Hui Guan is a Ph.D. candidate in the Department of Electrical and Computer Engineering, North Carolina State University, working with Dr. Xipeng Shen and Dr. Hamid Krim. Her research lies in the intersection between Machine Learning and Programming Systems, with a focus on improving Machine Learning (e.g., speed, scalability, reliability) through innovations in algorithms and programming systems (e.g., compilers, runtime), as well as leveraging Machine Learning to improve High-Performance Computing.