| Download | - View final version: Active measure reinforcement learning for observation cost minimization (PDF, 2.4 MiB)
|
|---|
| DOI | Resolve DOI: https://doi.org/10.21428/594757db.72846d04 |
|---|
| Author | Search for: Bellinger, Colin1; Search for: Coles, Rory; Search for: Crowley, Mark; Search for: Tamblyn, Isaac2 |
|---|
| Affiliation | - National Research Council of Canada. Digital Technologies
- National Research Council of Canada. Security and Disruptive Technologies
|
|---|
| Format | Text, Article |
|---|
| Conference | 34th Canadian Conference on Artificial Intelligence, May 25-28, 2021, Vancouver, British Columbia [Virtual Event] |
|---|
| Physical description | 12 p. |
|---|
| Subject | reinforcement learning; active learning; partial observability; sample efficiency |
|---|
| Abstract | Markov Decision Processes (MDP) with explicit measurement cost are a class of environments in which the agent learns to maximize the costed return. Here, we define the costed return as the discounted sum of rewards minus the sum of the explicit cost of measuring the next state. The RL agent can freely explore the relationship between actions and rewards but is charged each time it measures the next state. Thus, an optimal agent must learn a policy without making a large number of measurements. We propose the active measure RL framework (Amrl) as a solution to this novel class of problem, and contrast it with standard reinforcement learning under full observability and planning under partially observability. We demonstrate that Amrl-Q agents learn to shift from a reliance on costly measurements to exploiting a learned transition model in order to reduce the number of real-world measurements and achieve a higher costed return. Our results demonstrate the superiority of Amrl-Q over standard RL methods, Q-learning and Dyna-Q, and POMCP for planning under a POMDP in environments with explicit measurement costs. |
|---|
| Publication date | 2021-06-08 |
|---|
| Publisher | Canadian Artificial Intelligence Association |
|---|
| Licence | |
|---|
| In | |
|---|
| Language | English |
|---|
| Peer reviewed | Yes |
|---|
| Export citation | Export as RIS |
|---|
| Report a correction | Report a correction (opens in a new tab) |
|---|
| Record identifier | 0a738f55-7c86-4259-9a0a-a1a1e882e8c8 |
|---|
| Record created | 2021-12-13 |
|---|
| Record modified | 2021-12-14 |
|---|
