非均質馬可夫決策過程在演算法上的一些數學性質

Abstract

Hopp, Bean and Duenyas (1992) formulated a mixed integer program (MIP) to determine whether a finite time horizon is a forecast horizon in a nonhomogeneous Markov decision process (NMDP). Their formula is provided by complex Bender`s decomposition techniques. In this paper, we investigate in details of the contraction property and affine mapping property of NMDP, By these properties, Hopp et al.`s formula is relicvcd of the complex MIP formula and Bender`s decomposition algorithm. We only need to check a finite number of vertices at a polyhedral set shaped by the solution of the NMDP. The analysis gives insight into the NMDP and facilitates the process in detcnnining the forecast horizon. Furthermore, this NMDP formulation is presented in the form of a simple dynamic function which is different from the linear program presented by Hopp et al.