Textbook in PDF format

Vol-1 Foundations and Principles
Vol-2 Statistical Learning and Deep Learning
Today, mathematical methods, models, and computational algorithms are playing increasingly significant roles in addressing major challenges arising from scientific research and technological development. Although many novel methods and algorithms, such as deep learning and artificial intelligence, are emerging and reshaping various areas at an unprecedented pace, their core ideas and working mechanisms are inherently related to and deeply rooted in some essential mathematical foundations and principles. By performing an in-depth survey on the underlying foundations, principles, and algorithms, this book aims to navigate the vast landscape of mathematical methods widely used in diverse domains. This book starts with a survey of mathematical foundations, including essential concepts and theorems in real analysis, linear algebra, etc. Then it covers a broad spectrum of applied mathematical methods, ranging from traditional ones such as optimizations and dynamics modeling, to state-of-the-art such as machine learning, deep learning, and reinforcement learning. The emphasis is placed on methods regarding statistical modeling, stochastic and dynamical system modeling, optimal decision making, and statistical learning. For each part, this book organizes fundamental definitions, theorems, methods, and algorithms in a logical, self-explanatory way.
Preface
Mathematical foundations
Sets, sequences, and series.
Metric space.
Advanced calculus.
Linear algebra i : vector space and linear maps.
Linear algebra ii : matrix analysis.
Basic functional analysis.
Mathematical optimization methods
Unconstrained nonlinear optimization.
Constrained nonlinear optimization.
Linear optimization.
Convex analysis and convex optimization.
Classical statistical methods
Probability theory.
Statistical distributions.
Statistical estimation theory.
Multivariate statistical methods.
Linear regression analysis.
Monte-Carlo methods.
Dynamics modeling methods
Models and estimation in linear systems.
Estimation in dynamical systems.
Stochastic process.
Stochastic calculus.
Markov chain and random walk.
Time series analysis.
Statistical learning methods
Supervised learning principles.
Linear models for regression.
Linear models for classification.
Generative models.
K-nearest neighbors.
Tree methods.
Ensemble and boosting methods.
Unsupervised statistical learning.
Practical statistical learning.
Deep learning methods
Foundations of deep learning.
Network training and optimization.
Convolutional neural networks.
Recurrent neural networks.
Optimal control and reinforcement learning
Classical optimal control theory.
Reinforcement learning.
Applications
Natural language processing i : foundation.
Natural language processing ii : tasks.
Deep learning for automatic speech recognition.
Appendix
A supplemental mathematical facts.
Alphabetical Index

 
Textbook in PDF format

Vol-1 Foundations and Principles
Vol-2 Statistical Learning and Deep Learning
Today, mathematical methods, models, and computational algorithms are playing increasingly significant roles in addressing major challenges arising from scientific research and technological development. Although many novel methods and algorithms, such as deep learning and artificial intelligence, are emerging and reshaping various areas at an unprecedented pace, their core ideas and working mechanisms are inherently related to and deeply rooted in some essential mathematical foundations and principles. By performing an in-depth survey on the underlying foundations, principles, and algorithms, this book aims to navigate the vast landscape of mathematical methods widely used in diverse domains. This book starts with a survey of mathematical foundations, including essential concepts and theorems in real analysis, linear algebra, etc. Then it covers a broad spectrum of applied mathematical methods, ranging from traditional ones such as optimizations and dynamics modeling, to state-of-the-art such as machine learning, deep learning, and reinforcement learning. The emphasis is placed on methods regarding statistical modeling, stochastic and dynamical system modeling, optimal decision making, and statistical learning. For each part, this book organizes fundamental definitions, theorems, methods, and algorithms in a logical, self-explanatory way.
Preface
Mathematical foundations
Sets, sequences, and series.
Metric space.
Advanced calculus.
Linear algebra i : vector space and linear maps.
Linear algebra ii : matrix analysis.
Basic functional analysis.
Mathematical optimization methods
Unconstrained nonlinear optimization.
Constrained nonlinear optimization.
Linear optimization.
Convex analysis and convex optimization.
Classical statistical methods
Probability theory.
Statistical distributions.
Statistical estimation theory.
Multivariate statistical methods.
Linear regression analysis.
Monte-Carlo methods.
Dynamics modeling methods
Models and estimation in linear systems.
Estimation in dynamical systems.
Stochastic process.
Stochastic calculus.
Markov chain and random walk.
Time series analysis.
Statistical learning methods
Supervised learning principles.
Linear models for regression.
Linear models for classification.
Generative models.
K-nearest neighbors.
Tree methods.
Ensemble and boosting methods.
Unsupervised statistical learning.
Practical statistical learning.
Deep learning methods
Foundations of deep learning.
Network training and optimization.
Convolutional neural networks.
Recurrent neural networks.
Optimal control and reinforcement learning
Classical optimal control theory.
Reinforcement learning.
Applications
Natural language processing i : foundation.
Natural language processing ii : tasks.
Deep learning for automatic speech recognition.
Appendix
A supplemental mathematical facts.
Alphabetical Index