# Solutions of semi-Markov control models with recursive discount rates and approximation by $\u03f5$-optimal policies

Yofre H. García; Juan González-Hernández

Kybernetika (2019)

- Volume: 55, Issue: 3, page 495-517
- ISSN: 0023-5954

## Access Full Article

top## Abstract

top## How to cite

topH. García, Yofre, and González-Hernández, Juan. "Solutions of semi-Markov control models with recursive discount rates and approximation by $\epsilon $-optimal policies." Kybernetika 55.3 (2019): 495-517. <http://eudml.org/doc/294562>.

@article{H2019,

abstract = {This paper studies a class of discrete-time discounted semi-Markov control model on Borel spaces. We assume possibly unbounded costs and a non-stationary exponential form in the discount factor which depends of on a rate, called the discount rate. Given an initial discount rate the evolution in next steps depends on both the previous discount rate and the sojourn time of the system at the current state. The new results provided here are the existence and the approximation of optimal policies for this class of discounted Markov control model with non-stationary rates and the horizon is finite or infinite. Under regularity condition on sojourn time distributions and measurable selector conditions, we show the validity of the dynamic programming algorithm for the finite horizon case. By the convergence in finite steps to the value functions, we guarantee the existence of non-stationary optimal policies for the infinite horizon case and we approximate them using non-stationary $\epsilon $-optimal policies. We illustrated our results a discounted semi-Markov linear-quadratic model, when the evolution of the discount rate follows an appropriate type of stochastic differential equation.},

author = {H. García, Yofre, González-Hernández, Juan},

journal = {Kybernetika},

keywords = {optimal stochastic control; dynamic programming method; semi-Markov processes},

language = {eng},

number = {3},

pages = {495-517},

publisher = {Institute of Information Theory and Automation AS CR},

title = {Solutions of semi-Markov control models with recursive discount rates and approximation by $\epsilon $-optimal policies},

url = {http://eudml.org/doc/294562},

volume = {55},

year = {2019},

}

TY - JOUR

AU - H. García, Yofre

AU - González-Hernández, Juan

TI - Solutions of semi-Markov control models with recursive discount rates and approximation by $\epsilon $-optimal policies

JO - Kybernetika

PY - 2019

PB - Institute of Information Theory and Automation AS CR

VL - 55

IS - 3

SP - 495

EP - 517

AB - This paper studies a class of discrete-time discounted semi-Markov control model on Borel spaces. We assume possibly unbounded costs and a non-stationary exponential form in the discount factor which depends of on a rate, called the discount rate. Given an initial discount rate the evolution in next steps depends on both the previous discount rate and the sojourn time of the system at the current state. The new results provided here are the existence and the approximation of optimal policies for this class of discounted Markov control model with non-stationary rates and the horizon is finite or infinite. Under regularity condition on sojourn time distributions and measurable selector conditions, we show the validity of the dynamic programming algorithm for the finite horizon case. By the convergence in finite steps to the value functions, we guarantee the existence of non-stationary optimal policies for the infinite horizon case and we approximate them using non-stationary $\epsilon $-optimal policies. We illustrated our results a discounted semi-Markov linear-quadratic model, when the evolution of the discount rate follows an appropriate type of stochastic differential equation.

LA - eng

KW - optimal stochastic control; dynamic programming method; semi-Markov processes

UR - http://eudml.org/doc/294562

ER -

## References

top- Arnold, L., Stochastic Differential Equations., John Wiley and Sons, New York 1973. MR0443083
- Ash, R., Doléans-Dade, C., Probability and Measure Theory., Academic Press, San Diego, 2000. Zbl0944.60004MR1810041
- Bhattacharya, R., Majumdar, M., 10.1016/0378-3758(89)90053-0, J. Statist. Plann. Inference 21 (1989), 3, 365-381. MR0995606DOI10.1016/0378-3758(89)90053-0
- Bertsekas, D., Shreve, S., Stochastic Optimal Control: The Discrete Time Case., Athena Scientific, Belmont, Massachusetts 1996. Zbl0633.93001MR0809588
- Blackwell, D., 10.1214/aoms/1177700285, Ann. Math. Statist. 36, (1965), 226-235. MR0173536DOI10.1214/aoms/1177700285
- Cani, J. De, 10.1287/mnsc.10.4.716, Management. Sci. 10 (1963), 716-733. MR0169690DOI10.1287/mnsc.10.4.716
- Drenyovszki, R., Kovács, L., Tornai, K., Oláh, A., I., I. Pintér, 10.14736/kyb-2017-6-1100, Kybernetika 53 (2017), 6, 1100-1117. DOI10.14736/kyb-2017-6-1100
- Dekker, R., Hordijk, A., 10.1007/bf02055581, Ann. Oper. Res. 28 (1991), 185-212. MR1105173DOI10.1007/bf02055581
- García, Y., González-Hernández, J., 10.14736/kyb-2016-3-0403, Kybernetika 52 (2016), 403-426. MR3532514DOI10.14736/kyb-2016-3-0403
- González-Hernández, J., López-Martínez, R., Pérez-Hernández, J., 10.1007/s00186-006-0092-2, Math. Meth. Oper. Res. 65 (2006), 27-44. Zbl1126.90075MR2302022DOI10.1007/s00186-006-0092-2
- González-Hernández, J., Villarreal-Rodríguez, C., Optimal solutions of constrained discounted semi-Markov control problems
- Hernández-Lerma, O., Lasserre, J., 10.1007/978-1-4612-0729-0_1, Springer-Verlag, New York 1996. Zbl0840.93001MR1363487DOI10.1007/978-1-4612-0729-0_1
- Hu, Q., Yue, W., 10.14736/kyb-2017-1-0059, Springer-Verlag, Advances in Mechanics and Mathematics book series 14, (2008). MR2361223DOI10.14736/kyb-2017-1-0059
- Huang, X., Huang, Y., 10.14736/kyb-2017-1-0059, Kybernetika 53 (2017), 1, 59-81. MR3638556DOI10.14736/kyb-2017-1-0059
- Howard, R., Semi-Markovian decision processes., Bull. Int. Statist. Inst. 40 (1963), 2, 625-652. MR0173545
- Jewell, W., 10.1287/opre.11.6.938, Oper. Res. 11 (1963), 938-971. MR0163375DOI10.1287/opre.11.6.938
- Luque-Vázquez, F., Hernández-Lerma, O., 10.4064/am-26-3-315-331, Appl. Math. 26 (1999), 315-331. MR1726636DOI10.4064/am-26-3-315-331
- Luque-Vásquez, F., Minjárez-Sosa, J. A., 10.1007/s001860400406, Math. Methods Oper. Res. 61 (2005), 455-468. MR2225824DOI10.1007/s001860400406
- Luque-Vásquez, F., Minjárez-Sosa, J., Rosas, L., 10.1007/s00245-009-9086-9, Appl. Math. Optim. 61, (2010), 317-336. MR2609593DOI10.1007/s00245-009-9086-9
- Schweitzer, P., Perturbation Theory and Markovian Decision Processes., Ph.D. Dissertation, Massachusetts Institute of Technology, 1965. MR2939613
- Vasicek, O., 10.1016/0304-405x(77)90016-2, J. Financ. Econom. 5 (1977), 177-188. DOI10.1016/0304-405x(77)90016-2
- Vega-Amaya, O., Average optimatily in semi-Markov control models on Borel spaces: unbounded costs and control., Bol. Soc. Mat. Mexicana 38 (1997), 2, 47-60. MR1313106
- Zagst, R., 10.1007/BF01432360, ZOR - Math. Methods Oper. Res. 41 (1995), 277-288. MR1339493DOI10.1007/BF01432360

## NotesEmbed ?

topTo embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.