GAIG Game AI Research Group @ QMUL

Generalising Discrete Action Spaces with Conditional Action Trees

2021
Bamford, Christopher and Ovalle, Alvaro

Abstract

There are relatively few conventions followed in reinforcement learning (RL) environments to structure the action spaces. As a consequence the application of RL algorithms to tasks with large action spaces with multiple components require additional effort to adjust to different formats. In this paper we introduce {m Conditional Action Trees} with two main objectives: (1) as a method of structuring action spaces in RL to generalise across several action space specifications, and (2) to formalise a process to significantly reduce the action space by decomposing it into multiple sub-spaces, favoring a multi-staged decision making approach. We show several proof-of-concept experiments validating our scheme, ranging from environments with basic discrete action spaces to those with large combinatorial action spaces commonly found in RTS-style games.
Arxiv: https://arxiv.org/abs/2104.07294 

Cite this work

@article{bamford2021generalising,
author= {Bamford, Christopher and Ovalle, Alvaro},
title= {{Generalising Discrete Action Spaces with Conditional Action Trees}},
year= {2021},
journal= {{arXiv:2104.07294}},
abstract= {There are relatively few conventions followed in reinforcement learning (RL) environments to structure the action spaces. As a consequence the application of RL algorithms to tasks with large action spaces with multiple components require additional effort to adjust to different formats. In this paper we introduce {m Conditional Action Trees} with two main objectives: (1) as a method of structuring action spaces in RL to generalise across several action space specifications, and (2) to formalise a process to significantly reduce the action space by decomposing it into multiple sub-spaces, favoring a multi-staged decision making approach. We show several proof-of-concept experiments validating our scheme, ranging from environments with basic discrete action spaces to those with large combinatorial action spaces commonly found in RTS-style games.},
}

Comments

Content