Textbook in PDF format

Stochastic modeling is a set of quantitative techniques for analyzing practical systems with random factors. This area is highly technical and mainly developed by mathematicians. Most existing books are for those with extensive mathematical training; this book minimizes that need and makes the topics easily understandable.
Fundamentals of Stochastic Models offers many practical examples and applications and bridges the gap between elementary stochastics process theory and advanced process theory. It addresses both performance evaluation and optimization of stochastic systems and covers different modern analysis techniques such as matrix analytical methods and diffusion and fluid limit methods. It goes on to explore the linkage between stochastic models, Machine Learning, and Artificial Intelligence, and discusses how to make use of intuitive approaches instead of traditional theoretical approaches.
A dynamic optimization problem under uncertainty can be modeled as a Markov decision process (MDP). Such a problem involves a sequence of decisions made by an agent at different points in time over either a finite or an infinite time horizon. The MDP is a Markov process in which some decisions (also called actions) can be made for each state at an instant in time (called a decision epoch). The time interval between two consecutive decision epochs can be either deterministic or random. The formal is called a discrete-time MDP, denoted by DTMDP, and the latter is called a continuous-time MDP, denoted by CTMDP. We give two examples of these MDPs.
The goal is to minimize the mathematical background of readers that is required to understand the topics covered in this book. Thus, the book is appropriate for professionals and students in industrial engineering, business and economics, Computer Science, and applied mathematics