### Abstract

1. Most sequential decision-making problems in conservation can be viewed conceptually and modelled as a Markov decision process. The goal in this context is to construct a policy that associates each state of the system with a particular action. This policy should offer optimal performance in the sense of maximizing or minimizing a specified conservation objective 2. Dynamic programming algorithms rely on explicit enumeration to derive the optimal policy. This is problematic from a computational perspective as the size of the state space grows exponentially with the number of state variables. 3. We present a state aggregation method where the idea is to capture the most important aspects of the original Markov decision process, find an optimal policy over this reduced space and use this as an approximate solution to the original problem. 4. Applying the aggregation method to a species reintroduction problem, we demonstrate how we were able to reduce the number of states by 75% and reduce the size of the transition matrices by almost 94% (324 vs. 5184), and the abstract action matched the optimal action more than 86% of the time. 5. We conclude that the aggregation method is not a panacea for the curse of dimensionality, but it does advance our ability to construct approximately optimal policies in systems with large state spaces.

Original language | English (US) |
---|---|

Pages (from-to) | 949-957 |

Number of pages | 9 |

Journal | Methods in Ecology and Evolution |

Volume | 3 |

Issue number | 6 |

DOIs | |

State | Published - Dec 1 2012 |

### Fingerprint

### Keywords

- Abstraction
- Curse of dimensionality
- Markov decision process
- State space
- Stochastic dynamic programming

### ASJC Scopus subject areas

- Ecology, Evolution, Behavior and Systematics
- Ecological Modeling

### Cite this

*Methods in Ecology and Evolution*,

*3*(6), 949-957. https://doi.org/10.1111/j.2041-210X.2012.00242.x

**A simple method for dealing with large state spaces.** / Schapaugh, Adam W.; Tyre, Andrew J.

Research output: Contribution to journal › Article

*Methods in Ecology and Evolution*, vol. 3, no. 6, pp. 949-957. https://doi.org/10.1111/j.2041-210X.2012.00242.x

}

TY - JOUR

T1 - A simple method for dealing with large state spaces

AU - Schapaugh, Adam W.

AU - Tyre, Andrew J.

PY - 2012/12/1

Y1 - 2012/12/1

N2 - 1. Most sequential decision-making problems in conservation can be viewed conceptually and modelled as a Markov decision process. The goal in this context is to construct a policy that associates each state of the system with a particular action. This policy should offer optimal performance in the sense of maximizing or minimizing a specified conservation objective 2. Dynamic programming algorithms rely on explicit enumeration to derive the optimal policy. This is problematic from a computational perspective as the size of the state space grows exponentially with the number of state variables. 3. We present a state aggregation method where the idea is to capture the most important aspects of the original Markov decision process, find an optimal policy over this reduced space and use this as an approximate solution to the original problem. 4. Applying the aggregation method to a species reintroduction problem, we demonstrate how we were able to reduce the number of states by 75% and reduce the size of the transition matrices by almost 94% (324 vs. 5184), and the abstract action matched the optimal action more than 86% of the time. 5. We conclude that the aggregation method is not a panacea for the curse of dimensionality, but it does advance our ability to construct approximately optimal policies in systems with large state spaces.

AB - 1. Most sequential decision-making problems in conservation can be viewed conceptually and modelled as a Markov decision process. The goal in this context is to construct a policy that associates each state of the system with a particular action. This policy should offer optimal performance in the sense of maximizing or minimizing a specified conservation objective 2. Dynamic programming algorithms rely on explicit enumeration to derive the optimal policy. This is problematic from a computational perspective as the size of the state space grows exponentially with the number of state variables. 3. We present a state aggregation method where the idea is to capture the most important aspects of the original Markov decision process, find an optimal policy over this reduced space and use this as an approximate solution to the original problem. 4. Applying the aggregation method to a species reintroduction problem, we demonstrate how we were able to reduce the number of states by 75% and reduce the size of the transition matrices by almost 94% (324 vs. 5184), and the abstract action matched the optimal action more than 86% of the time. 5. We conclude that the aggregation method is not a panacea for the curse of dimensionality, but it does advance our ability to construct approximately optimal policies in systems with large state spaces.

KW - Abstraction

KW - Curse of dimensionality

KW - Markov decision process

KW - State space

KW - Stochastic dynamic programming

UR - http://www.scopus.com/inward/record.url?scp=84871014380&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84871014380&partnerID=8YFLogxK

U2 - 10.1111/j.2041-210X.2012.00242.x

DO - 10.1111/j.2041-210X.2012.00242.x

M3 - Article

AN - SCOPUS:84871014380

VL - 3

SP - 949

EP - 957

JO - Methods in Ecology and Evolution

JF - Methods in Ecology and Evolution

SN - 2041-210X

IS - 6

ER -