Date of Award
Master of Science
Department of Operational Sciences
Jeffrey P. Kharoufeh, PhD
In this thesis, the problem of computing the cumulative distribution function (cdf) of the random time required for a system to first reach a specified reward threshold when the rate at which the reward accrues is controlled by a continuous time stochastic process is considered. This random time is a type of first passage time for the cumulative reward process. The major contribution of this work is a simplified, analytical expression for the Laplace-Stieltjes Transform of the cdf in one dimension rather than two. The result is obtained using two techniques: i) by converting an existing partial differential equation to an ordinary differential equation with a known solution, and ii) by inverting an existing two-dimensional result with respect to one of the dimensions. The results are applied to a variety of real-world operational problems using one-dimensional numerical Laplace inversion techniques and compared to solutions obtained from numerical inversion of a two-dimensional transform, as well as those from Monte-Carlo simulation. Inverting one-dimensional transforms is computationally more expedient than inverting two-dimensional transforms, particularly as the number of states in the governing Markov process increases. The numerical results demonstrate the accuracy with which the one-dimensional result approximates the first passage time probabilities in a comparatively negligible amount of the time.
DTIC Accession Number
Sipe, Jeffrey A., "Transient Analysis and Applications of Markov Reward Processes" (2003). Theses and Dissertations. 4161.