Open this publication in new window or tab >> Show others...
2024 (English) In: Robotics and Autonomous Systems, ISSN 0921-8890, E-ISSN 1872-793X, Vol. 179, article id 104731Article in journal (Refereed) Published
Abstract [en] We explore sim-to-real transfer of deep reinforcement learning controllers for a heavy vehicle with active suspensions designed for traversing rough terrain. While related research primarily focuses on lightweight robots with electric motors and fast actuation, this study uses a forestry vehicle with a complex hydraulic driveline and slow actuation. We simulate the vehicle using multibody dynamics and apply system identification to find an appropriate set of simulation parameters. We then train policies in simulation using various techniques to mitigate the sim-to-real gap, including domain randomization, action delays, and a reward penalty to encourage smooth control. In reality, the policies trained with action delays and a penalty for erratic actions perform nearly at the same level as in simulation. In experiments on level ground, the motion trajectories closely overlap when turning to either side, as well as in a route tracking scenario. When faced with a ramp that requires active use of the suspensions, the simulated and real motions are in close alignment. This shows that the actuator model together with system identification yields a sufficiently accurate model of the actuators. We observe that policies trained without the additional action penalty exhibit fast switching or bang–bang control. These present smooth motions and high performance in simulation but transfer poorly to reality. We find that policies make marginal use of the local height map for perception, showing no indications of predictive planning. However, the strong transfer capabilities entail that further development concerning perception and performance can be largely confined to simulation.
Place, publisher, year, edition, pages
Elsevier, 2024
National Category
Electrical Engineering, Electronic Engineering, Information Engineering Other Physics Topics
Research subject
Physics; computer and systems sciences
Identifiers urn:nbn:se:umu:diva-226893 (URN) 10.1016/j.robot.2024.104731 (DOI) 2-s2.0-85196769514 (Scopus ID)
Projects Mistra Digital Forest
Funder Mistra - The Swedish Foundation for Strategic Environmental Research, Grant DIA 2017/14 #6Wallenberg AI, Autonomous Systems and Software Program (WASP)
2024-06-232024-06-232024-07-03 Bibliographically approved