Neural Stochastic Differential Equations for Probabilistic Modeling - (Machine Learning in Science, Technology, Engineering and Mathematics)

About the Book

This series is essential to meet the growing demand for structured and accessible resources that guide the integration of machine learning (ML) into engineering and technology. The series will provide a comprehensive resource that bridges the gap between theory and practice, making it invaluable for students, researchers, and professionals looking to leverage ML in their work.

Book Synopsis

Stochastic differential equations (SDEs) offer a natural framework for probabilistic modeling of high-dimensional data, and underlie much progress in generative AI. This book presents a scalable framework for SDEs in machine learning, developed by the authors: training neural networks to approximate stochastic processes, and performing tractable inference and sampling via efficient SDE solvers. We introduce approaches to learning SDEs for probabilistic modeling, including fitting non-linear SDEs with neural networks, and a novel model that learns SDE dynamics under given constraints on its marginal probability. We also develop an efficient algorithm for sampling from high-dimensional SDEs, effective for generating diverse, high-fidelity images and videos. Written for researchers and students in computer science, engineering, and mathematics, this book offers a principled, unified treatment of the mathematical foundations behind techniques now standard in generative modeling.

About the Author

Dr. Qinsheng Zhang obtained his Ph.D. in Robotics from the Georgia Institute of Technology in 2023, under the guidance of Yongxin Chen. He earned his BS degree in mechanical engineering from Shanghai Jiao Tong University in 2019. His research centers on probabilistic modeling of high-dimensional data and its application across various domains. Dr. Zhang investigates complex distributions underlying data, such as real images and feasible robot trajectories, emphasizing representation, learning, and sampling techniques. His research interests include density models, probabilistic robotics, and optimal transport. Notable recent publications in 2023 encompass Loss-Guided Diffusion Models, DiffCollage, gDDIM, and Fast Sampling of Diffusion Models with Exponential Integrator. Dr. Zhang curently serves as a research scientist at NVIDIA.

Dr. Yongxin Chen is an Associate Professor in the School of Aerospace Engineering at Georgia Institute of Technology. He previously served on the faculty at Iowa State University (2017-2018). Prior to that, he spent one year (2016-2017) at the Memorial Sloan Kettering Cancer Center (MSKCC) as a postdoctoral fellow. He received his BS from Shanghai Jiao Tong University in 2011, and Ph.D. from University of Minnesota in 2016, both in mechanical engineering. He is an awardee of the George S. Axelby Best Paper Award of IEEE Transactions on Automatic Control in 2017 and the best paper prize of SIAM Journal on Control and Optimization in 2023. He received the NSF Faculty Early Career Development Program (CAREER) Award in 2020, the Simons-Berkeley Research Fellowship in 2021, the A. V. 'Bal' Balakrishnan Award in 2021, and the Donald P. Eckman Award for outstanding young engineer in the field of automatic control in 2022. His current research interests are in the areas of control theory, machine learning, and robotics. He enjoys developing new algorithms and theoretical frameworks for real world applications.