| Download | - View accepted manuscript: Probabilistic Exploration in Planning while Learning (PDF, 214 KiB)
|
|---|
| Author | Search for: Karakoulas, G. |
|---|
| Format | Text, Article |
|---|
| Conference | Eleventh International Conference on Uncertainty in Artificial Intelligence (UAI'95), August 18-20, 1995, Montreal, Quebec, Canada |
|---|
| Subject | sequential decision-making; exploration; probabilistic hill-climbing; Q-learning; exploration; escalade probabiliste; apprentissage-Q |
|---|
| Abstract | Sequential decision tasks with incomplete information are characterized by the exploration problem; namely the trade-off between further exploration for learning more about the environment and immediate exploitation of the accrued information for decision-making. Within artificial intelligence, there has been an increasing interest in studying planning-while-learning algorithms for these decision tasks. In this paper we focus on the exploration problem in reinforcement learning and Q-learning in particular. The existing exploration strategies for Q-learning are of a heuristic nature and they exhibit limited scalability in tasks with large (or infinite) state and action spaces. Efficient experimentation is needed for resolving uncertainties when possible plans are compared (i.e. exploration). The experimentation should be sufficient for selecting with statistical significance a locally optimal plan (i.e. exploitation). For this purpose, we develop a probabilistic hill-climbing algorithm that uses a statistical selection procedure to decide how much exploration is needed for selecting a plan which is, with arbitrarily high probability, arbitrarily close to a locally optimal one. Due to its generality the algorithm can be employed for the exploration strategy of robust Q-learning. An experiment on a relatively complex control task shows that the proposed exploration strategy performs better than a typical exploration strategy. |
|---|
| Publication date | 1995 |
|---|
| In | |
|---|
| Language | English |
|---|
| NRC number | NRCC 38386 |
|---|
| NPARC number | 8913955 |
|---|
| Export citation | Export as RIS |
|---|
| Report a correction | Report a correction (opens in a new tab) |
|---|
| Record identifier | d86e1142-21a8-4c51-954b-53899bd30f9e |
|---|
| Record created | 2009-04-22 |
|---|
| Record modified | 2020-04-29 |
|---|
