Markov Decision Processes Toolbox

Description

The Markov Decision Processes (MDP) toolbox proposes functions related to the resolution of discrete-time Markov Decision Processes: finite horizon, value iteration, policy iteration, linear programming algorithms with some variants and also proposes some functions related to Reinforcement Learning.

Details

Author(s)

Iadine Chadès <Iadine.Chades@csiro.au>
Guillaume Chapron <gchapron@carnivoreconservation.org>
Marie-Josée Cros <Marie-Josee.Cros@toulouse.inra.fr>
Fredérick Garcia <fgarcia@toulouse.inra.fr>
Régis Sabbadin <Regis.Sabbadin@toulouse.inra.fr>

References

Chadès, I., Chapron, G., Cros, M.-J., Garcia, F. & Sabbadin, R. 2014. MDPtoolbox: a multi-platform toolbox to solve stochastic dynamic programming problems. Ecography DOI:10.1111/ecog.00888
Puterman, M. L. 1994. Markov Decision Processes. John Wiley & Sons, New-York.

Examples

# Generates a random MDP problem
set.seed(0)
mdp_example_rand(2, 2)
mdp_example_rand(2, 2, FALSE)
mdp_example_rand(2, 2, TRUE)
mdp_example_rand(2, 2, FALSE, matrix(c(1,0,1,1),2,2))

# Generates a MDP for a simple forest management problem
MDP <- mdp_example_forest()

# Find an optimal policy
results <- mdp_policy_iteration(MDP$P, MDP$R, 0.9)

# Visualise the policy
results$policy