| DOI | Resolve DOI: https://doi.org/10.1115/OMAE2025-157355 |
|---|
| Author | Search for: Amarappulige, Charuka; Search for: Imtiaz, Syed; Search for: Ahmed, Salim; Search for: Islam, Shameem1ORCID identifier: https://orcid.org/0000-0002-2129-5333; Search for: Zaman, Hasanat1 |
|---|
| Affiliation | - National Research Council Canada. Ocean, Coastal and River Engineering
|
|---|
| Format | Text, Article |
|---|
| Conference | ASME 2025 44th International Conference on Ocean, Offshore and Arctic Engineering, June 22-27, 2025, Vancouver, British Columbia, Canada |
|---|
| Subject | nonlinear model predictive controller (NMPC); reinforcement learning (RL); deep deterministic policy gradient (DDPG); autonomous surface vessel (ASV); control equipment; reinforcement learning; unmanned surface vehicles |
|---|
| Abstract | For autonomous surface vessels, conventional model-based control strategies face challenges due to the inherent uncertainties and inaccuracies associated with the model. This study investigates the application of reinforcement learning (RL) controllers, specifically using the Deep Deterministic Policy Gradient (DDPG) algorithm, to reduce dependency on model accuracy by enabling controllers to learn through experience. To further enhance RL controller performance, this work integrates Nonlinear Model Predictive Control (NMPC) for assisted training.
The methodology involves developing a DDPG-based RL controller for the approximated test vessel’s dynamics and training the RL controller for point tracking. Observations reveal that, while the RL controller successfully learns the control actions required for point tracking, the generated actions lack the smoothness necessary for real-world applications. To address this, NMPC is introduced to assist the RL controller in optimizing control actions during training, yielding results that are more suitable for practical implementation.
The findings demonstrate that NMPC assisted DDPG controller can effectively learn vessel dynamics through iterative training, reducing the reliance on precise vessel models. Additionally, the integration of NMPC enhances control action quality, and it has been validated with performance indicators via iterative simulations data. The control variance measuring indicators showed a reduction of an average 60% compared to conventional DDPG controller and the tracking accuracy increase compared to NMPC by 58% on average. This work paves the way for a potential combination of established controllers with RL controller in order to enhance accuracy and performance. |
|---|
| Publication date | 2025-06-22 |
|---|
| Publisher | American Society of Mechanical Engineers |
|---|
| In | |
|---|
| Language | English |
|---|
| Peer reviewed | Yes |
|---|
| Export citation | Export as RIS |
|---|
| Report a correction | Report a correction (opens in a new tab) |
|---|
| Record identifier | 60096670-c232-416f-985e-a28e4ec2998e |
|---|
| Record created | 2025-09-24 |
|---|
| Record modified | 2025-09-24 |
|---|
