非均質馬可夫決策系統的決策空間

Abstract

在求無限期非均質馬可夫決策過程（nonhomogeneous Markov decisinon processes）第一期的的最佳解時，我們通常要將它表示成有限期的動態規劃問題。動態規劃可以用合成函數型式表示，也可以用最常見的線性規劃型式表示。

Hopp, Bean and Duenyas(1992) formulate a mixed integer program (MIP) to determine whether a finite time horizon is a forecast horizon in a nonhomogeneous Markov decision process(NMDP). Their formula are solved by complex Bender`s decomposition In this thesis, we make an examination in details of the contraction property and affine mapping property of NMDP. By these properties we are relieved of the complex MIP formula and Bender`s decomposition algorithm. The main contribution of the thesis is to show that it is not necessary to determine the optimal policies by running through the whole feasible solution space of their MIP problem. We only need to check a finite number of vertices at a polyhedral set shaped by the solution of the NMDP. The analysis shows insights into the NMDP and facilitate the prosess in determining the forecast horizon. Furthermore, this NMDP formulation is presented in the form of a simple dynamic function which is different from the linear program presented by Hopp, Bean and Duenyas.