Deep reinforcement learning has achieved great successes in recent years, however, one main challenge is the sample inefficiency. In this paper, we focus on how to use action guidance by means of a non-expert demonstrator to improve sample efficiency in a domain with sparse, delayed, and possibly deceptive rewards: the recently-proposed multi-agent benchmark of Pommerman. We propose a new framework where even a non-expert simulated demonstrator, e.g., planning algorithms such as Monte Carlo tree search with a small number rollouts, can be integrated within asynchronous distributed deep reinforcement learning methods. Compared to a vanilla deep RL algorithm, our proposed methods both learn faster and converge to better policies on a two-player mini version of the Pommerman game.
Bibtex
@inproceedings{kartal2019action,
title={{Action Guidance with MCTS for Deep Reinforcement Learning}},
author={Bilal Kartal and Pablo Hernandez-Leal and Matthew E. Taylor},
year={2019},
booktitle={AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment},
}
Related Research
-
Leveraging Environment Interaction for Automated PDDL Generation and Planning with Large Language Models
Leveraging Environment Interaction for Automated PDDL Generation and Planning with Large Language Models
S. Mahdavi, R. Aoki, K. Tang, and Y. Cao. Conference on Neural Information Processing System (NeurIPS)
Publications
-
Jump Starting Bandits with LLM-Generated Prior Knowledge
Jump Starting Bandits with LLM-Generated Prior Knowledge
P. A. Alamdari, Y. Cao, and K. Wilson. Conference on Empirical Methods in Natural Language Processing
Publications
-
Our NeurIPS 2021 Reading List
Our NeurIPS 2021 Reading List
Y. Cao, K. Y. C. Lui, T. Durand, J. He, P. Xu, N. Mehrasa, A. Radovic, A. Lehrmann, R. Deng, A. Abdi, M. Schlegel, and S. Liu.
Computer Vision; Data Visualization; Graph Representation Learning; Learning And Generalization; Natural Language Processing; Optimization; Reinforcement Learning; Time series Modelling; Unsupervised Learning
Research