Solving infinite-horizon POMDPs with memoryless stochastic policies in state-action space
- Authors: Johannes Müller, Guido Montúfar
- Preprint year: 2022
- First public date: 2022-05-27
- arXiv: 2205.14098
- Status: Published
- Publication type: Extended abstract
- Publication year: 2022
- Proceedings: Reinforcement Learning and Decision Making (RLDM 2022), extended abstract
Abstract
Reward optimization in fully observable Markov decision processes is equivalent to a linear program over the polytope of state-action frequencies. Taking a similar perspective in the case of partially observable Markov decision processes with memoryless stochastic policies, the problem was recently formulated as the optimization of a linear objective subject to polynomial constraints. Based on this we present an approach for Reward Optimization in State-Action space (ROSA). We test this approach experimentally in maze navigation tasks. We find that ROSA is computationally efficient and can yield stability improvements over other existing methods.