Umeå University's logo

umu.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Control of rough terrain vehicles using deep reinforcement learning
Umeå University, Faculty of Science and Technology, Department of Physics.
Umeå University, Faculty of Science and Technology, Department of Physics.
Swedish University of Agricultural Sciences.
Umeå University, Faculty of Science and Technology, Department of Physics.ORCID iD: 0000-0002-0787-4988
2022 (English)In: IEEE Robotics and Automation Letters, E-ISSN 2377-3766, Vol. 7, no 1, p. 390-397Article in journal (Refereed) Published
Abstract [en]

We explore the potential to control terrain vehicles using deep reinforcement in scenarios where human operators and traditional control methods are inadequate. This letter presents a controller that perceives, plans, and successfully controls a 16-tonne forestry vehicle with two frame articulation joints, six wheels, and their actively articulated suspensions to traverse rough terrain. The carefully shaped reward signal promotes safe, environmental, and efficient driving, which leads to the emergence of unprecedented driving skills. We test learned skills in a virtual environment, including terrains reconstructed from high-density laser scans of forest sites. The controller displays the ability to handle obstructing obstacles, slopes up to 27 degrees, and a variety of natural terrains, all with limited wheel slip, smooth, and upright traversal with intelligent use of the active suspensions. The results confirm that deep reinforcement learning has the potential to enhance control of vehicles with complex dynamics and high-dimensional observation data compared to human operators or traditional control methods, especially in rough terrain.

Place, publisher, year, edition, pages
IEEE, 2022. Vol. 7, no 1, p. 390-397
Keywords [en]
Artificial Intelligence, Control and Optimization, Computer Science Applications, Computer Vision and Pattern Recognition, Mechanical Engineering, Human-Computer Interaction, Biomedical Engineering, Control and Systems Engineering
National Category
Robotics Computer Vision and Robotics (Autonomous Systems) Other Physics Topics
Research subject
Physics; Computer Science; Automatic Control
Identifiers
URN: urn:nbn:se:umu:diva-189485DOI: 10.1109/lra.2021.3126904ISI: 000721999500008Scopus ID: 2-s2.0-85119416792OAI: oai:DiVA.org:umu-189485DiVA, id: diva2:1610874
Funder
Mistra - The Swedish Foundation for Strategic Environmental Research, 2017/14 #6eSSENCE - An eScience CollaborationSwedish National Infrastructure for Computing (SNIC), 2021/5-234Available from: 2021-11-12 Created: 2021-11-12 Last updated: 2024-01-17Bibliographically approved
In thesis
1. Terrain machine learning
Open this publication in new window or tab >>Terrain machine learning
2023 (English)Doctoral thesis, comprehensive summary (Other academic)
Alternative title[sv]
Maskininlärning i terräng
Abstract [en]

The use of heavy vehicles in rough terrain is vital in the industry but has negative implications for the climate and ecosystem. In addition, the demand for improved efficiency underscores the need to enhance these vehicles' navigation capabilities. Navigating rough terrain presents distinct challenges, including deformable soil, surface roughness, and spatial and temporal terrain variability. Focusing on forestry, this thesis aims to improve navigation using machine learning and physics simulations. Without considering the vehicle-terrain dynamics, methods for navigation can result in unsafe or unnecessarily challenging situations. Specifically, we address route planning, control for autonomous vehicles, and soilde formations. We simulate soil using the discrete element method and vehicles using multibody dynamics.

To enhance route planning, we train a predictor model that uses a height map of the terrain to predict measures of traversability. The model has a directional dependency, couples geometric terrain features with vehicle design and dynamics, and allows for swift evaluations over large areas. The proposed method facilitates detailed route planning, using multiple objectives to yield efficient solutions.

We address autonomy in rough terrain navigation by training a controller through deep reinforcement learning. The control policy uses a local height map for perception to plan and control a forwarder with actively articulated suspensions. The controller adapts to overcome various obstacles and demonstrates skilled driving in rough terrain.

Extending beyond simulation, we address the simulation-to-reality gap of vehicles with complex hydraulic drivelines through system identification and domain randomization. The results show that having an accurate model of the actuators, modelling system delays, and preventing bang-bang control yields successful transfer. Controllers that train in simulation and transfer to reality are a step toward autonomous vehicles.

While the previously mentioned studies assume rigid terrain, we also answer if the discrete element method can capture large soil deformations due to heavy traffic. The results show that the discrete element method can represent a wide variety of natural soil and that the resulting rut depths agree well with empirical models and experimental data, including multipass scenarios.

Place, publisher, year, edition, pages
Umeå: Umeå University, 2023. p. 38
Keywords
multibody dynamics simulation, rough terrain vehicle, autonomous vehicles, robotics control, discrete element method, sim-to-real, reinforcement learning
National Category
Robotics Other Physics Topics Forest Science Computer Vision and Robotics (Autonomous Systems)
Research subject
Physics
Identifiers
urn:nbn:se:umu:diva-207982 (URN)978-91-8070-060-3 (ISBN)978-91-8070-059-7 (ISBN)
Public defence
2023-06-01, NAT.D.410, Umeå, 09:00 (English)
Opponent
Supervisors
Funder
Mistra - The Swedish Foundation for Strategic Environmental Research, DIA 2017/14 #6
Available from: 2023-05-11 Created: 2023-05-05 Last updated: 2023-05-08Bibliographically approved

Open Access in DiVA

fulltext(2927 kB)386 downloads
File information
File name FULLTEXT02.pdfFile size 2927 kBChecksum SHA-512
3bc7460980989e7c3ed2229549d692f356a7a12a8e1bccd0f3c95996d0a5077c62b8a33cb26d6f84a78e94df95c833df62db01b089e634473179c970565c4670
Type fulltextMimetype application/pdf

Other links

Publisher's full textScopus

Authority records

Wiberg, ViktorWallin, ErikServin, Martin

Search in DiVA

By author/editor
Wiberg, ViktorWallin, ErikServin, Martin
By organisation
Department of Physics
In the same journal
IEEE Robotics and Automation Letters
RoboticsComputer Vision and Robotics (Autonomous Systems)Other Physics Topics

Search outside of DiVA

GoogleGoogle Scholar
Total: 397 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 436 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf