Exploiting Additive Structure in Factored MDPs for Reinforcement Learning

Thomas Degris , Olivier Sigaud (1) , Pierre-Henri Wuillemin (2)

1 Animatlab
2 DECISION

sdyna is a framework able to address large, discrete and stochastic reinforcement learning problems. It incrementally learns a fmdp representing the problem to solve while using fmdp planning techniques to build an efficient policy. spiti, an instantiation of sdyna, uses a planning method based on dynamic programming which cannot exploit the additive structure of a fmdp. In this paper, we present two new instantiations of sdyna, namely ulp and unatlp, using a linear programming based planning method that can exploit the additive structure of a fmdp and address problems out of reach of spiti.

Domaines: Informatique [cs]

Thomas Degris, Olivier Sigaud, Pierre-Henri Wuillemin. Exploiting Additive Structure in Factored MDPs for Reinforcement Learning. European Workshop on Reinforcement Learning, Jun 2008, Villeneuve d’Ascq, France. pp.15-26, ⟨10.1007/978-3-540-89722-4_2⟩. ⟨hal-01302178⟩