Adv. Math. Econ. 6, 55-68 (2004)

Recursive methods in probability control

Seiichi Iwamoto

Department of Economic Engineering, Graduate School of Economics, Kyushu University 27, Hakozaki 6-19-1, Higashiku, Fukuoka 812-8581, Japan

Received: April 14,2003 Revised: July 22,2003

JEL classification: C61, D81

Mathematical Subject Classification (2000): 90C39, 90C40, 90A43

Abstract. In this paper we propose recursive methods in probability control. We maximize the threshold probability that a “total” reward is greater than or equal to a given lower level. We take three types of reward function — general function, forward recursive function and backward recursive function. By three types of imbedding method, we derive a backward recursive equation for maximum probability function. The first is to view any history (alternating sequence of state and decision) as a new state. The second is to incorporate past values (cumulative rewards) in state dynamics. The third is a state expansion by attachment of future values (threshold levels). It is shown that the threshold probability problem has an optimal policy in a common large class.

Key words: recursive function, probability control, dynamic programming, invariant imbedding, forward recursive, backward recursive, backward recursive formula, forward recursive formula, past values, cumulative rewards, future values, threshold levels, expanded Markov policy, general policy, primitive policy