Umeå universitets logga

umu.sePublikationer
Ändra sökning
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
Control of rough terrain vehicles using deep reinforcement learning
Umeå universitet, Teknisk-naturvetenskapliga fakulteten, Institutionen för fysik.
Umeå universitet, Teknisk-naturvetenskapliga fakulteten, Institutionen för fysik.
Swedish University of Agricultural Sciences.
Umeå universitet, Teknisk-naturvetenskapliga fakulteten, Institutionen för fysik.ORCID-id: 0000-0002-0787-4988
2022 (Engelska)Ingår i: IEEE Robotics and Automation Letters, E-ISSN 2377-3766, Vol. 7, nr 1, s. 390-397Artikel i tidskrift (Refereegranskat) Published
Abstract [en]

We explore the potential to control terrain vehicles using deep reinforcement in scenarios where human operators and traditional control methods are inadequate. This letter presents a controller that perceives, plans, and successfully controls a 16-tonne forestry vehicle with two frame articulation joints, six wheels, and their actively articulated suspensions to traverse rough terrain. The carefully shaped reward signal promotes safe, environmental, and efficient driving, which leads to the emergence of unprecedented driving skills. We test learned skills in a virtual environment, including terrains reconstructed from high-density laser scans of forest sites. The controller displays the ability to handle obstructing obstacles, slopes up to 27 degrees, and a variety of natural terrains, all with limited wheel slip, smooth, and upright traversal with intelligent use of the active suspensions. The results confirm that deep reinforcement learning has the potential to enhance control of vehicles with complex dynamics and high-dimensional observation data compared to human operators or traditional control methods, especially in rough terrain.

Ort, förlag, år, upplaga, sidor
IEEE, 2022. Vol. 7, nr 1, s. 390-397
Nyckelord [en]
Artificial Intelligence, Control and Optimization, Computer Science Applications, Computer Vision and Pattern Recognition, Mechanical Engineering, Human-Computer Interaction, Biomedical Engineering, Control and Systems Engineering
Nationell ämneskategori
Robotik och automation Datorgrafik och datorseende Annan fysik
Forskningsämne
fysik; datalogi; reglerteknik
Identifikatorer
URN: urn:nbn:se:umu:diva-189485DOI: 10.1109/lra.2021.3126904ISI: 000721999500008Scopus ID: 2-s2.0-85119416792OAI: oai:DiVA.org:umu-189485DiVA, id: diva2:1610874
Forskningsfinansiär
Mistra - Stiftelsen för miljöstrategisk forskning, 2017/14 #6eSSENCE - An eScience CollaborationSwedish National Infrastructure for Computing (SNIC), 2021/5-234Tillgänglig från: 2021-11-12 Skapad: 2021-11-12 Senast uppdaterad: 2025-02-05Bibliografiskt granskad
Ingår i avhandling
1. Terrain machine learning
Öppna denna publikation i ny flik eller fönster >>Terrain machine learning
2023 (Engelska)Doktorsavhandling, sammanläggning (Övrigt vetenskapligt)
Alternativ titel[sv]
Maskininlärning i terräng
Abstract [en]

The use of heavy vehicles in rough terrain is vital in the industry but has negative implications for the climate and ecosystem. In addition, the demand for improved efficiency underscores the need to enhance these vehicles' navigation capabilities. Navigating rough terrain presents distinct challenges, including deformable soil, surface roughness, and spatial and temporal terrain variability. Focusing on forestry, this thesis aims to improve navigation using machine learning and physics simulations. Without considering the vehicle-terrain dynamics, methods for navigation can result in unsafe or unnecessarily challenging situations. Specifically, we address route planning, control for autonomous vehicles, and soilde formations. We simulate soil using the discrete element method and vehicles using multibody dynamics.

To enhance route planning, we train a predictor model that uses a height map of the terrain to predict measures of traversability. The model has a directional dependency, couples geometric terrain features with vehicle design and dynamics, and allows for swift evaluations over large areas. The proposed method facilitates detailed route planning, using multiple objectives to yield efficient solutions.

We address autonomy in rough terrain navigation by training a controller through deep reinforcement learning. The control policy uses a local height map for perception to plan and control a forwarder with actively articulated suspensions. The controller adapts to overcome various obstacles and demonstrates skilled driving in rough terrain.

Extending beyond simulation, we address the simulation-to-reality gap of vehicles with complex hydraulic drivelines through system identification and domain randomization. The results show that having an accurate model of the actuators, modelling system delays, and preventing bang-bang control yields successful transfer. Controllers that train in simulation and transfer to reality are a step toward autonomous vehicles.

While the previously mentioned studies assume rigid terrain, we also answer if the discrete element method can capture large soil deformations due to heavy traffic. The results show that the discrete element method can represent a wide variety of natural soil and that the resulting rut depths agree well with empirical models and experimental data, including multipass scenarios.

Ort, förlag, år, upplaga, sidor
Umeå: Umeå University, 2023. s. 38
Nyckelord
multibody dynamics simulation, rough terrain vehicle, autonomous vehicles, robotics control, discrete element method, sim-to-real, reinforcement learning
Nationell ämneskategori
Robotik och automation Annan fysik Skogsvetenskap Datorgrafik och datorseende
Forskningsämne
fysik
Identifikatorer
urn:nbn:se:umu:diva-207982 (URN)978-91-8070-060-3 (ISBN)978-91-8070-059-7 (ISBN)
Disputation
2023-06-01, NAT.D.410, Umeå, 09:00 (Engelska)
Opponent
Handledare
Forskningsfinansiär
Mistra - Stiftelsen för miljöstrategisk forskning, DIA 2017/14 #6
Tillgänglig från: 2023-05-11 Skapad: 2023-05-05 Senast uppdaterad: 2025-02-05Bibliografiskt granskad

Open Access i DiVA

fulltext(2927 kB)520 nedladdningar
Filinformation
Filnamn FULLTEXT02.pdfFilstorlek 2927 kBChecksumma SHA-512
3bc7460980989e7c3ed2229549d692f356a7a12a8e1bccd0f3c95996d0a5077c62b8a33cb26d6f84a78e94df95c833df62db01b089e634473179c970565c4670
Typ fulltextMimetyp application/pdf

Övriga länkar

Förlagets fulltextScopus

Person

Wiberg, ViktorWallin, ErikServin, Martin

Sök vidare i DiVA

Av författaren/redaktören
Wiberg, ViktorWallin, ErikServin, Martin
Av organisationen
Institutionen för fysik
I samma tidskrift
IEEE Robotics and Automation Letters
Robotik och automationDatorgrafik och datorseendeAnnan fysik

Sök vidare utanför DiVA

GoogleGoogle Scholar
Totalt: 557 nedladdningar
Antalet nedladdningar är summan av nedladdningar för alla fulltexter. Det kan inkludera t.ex tidigare versioner som nu inte längre är tillgängliga.

doi
urn-nbn

Altmetricpoäng

doi
urn-nbn
Totalt: 534 träffar
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf