Umeå University's logo

umu.sePublications
Change search
Link to record
Permanent link

Direct link
Publications (8 of 8) Show all publications
Wallin, E., Wiberg, V. & Servin, M. (2023). Multi-log grasping using reinforcement learning and virtual visual servoing. Robotics, 13(1), Article ID 3.
Open this publication in new window or tab >>Multi-log grasping using reinforcement learning and virtual visual servoing
2023 (English)In: Robotics, E-ISSN 2218-6581, Vol. 13, no 1, article id 3Article in journal (Refereed) Published
Abstract [en]

We explore multi-log grasping using reinforcement learning and virtual visual servoing for automated forwarding in a simulated environment. Automation of forest processes is a major challenge, and many techniques regarding robot control pose different challenges due to the unstructured and harsh outdoor environment. Grasping multiple logs involves various problems of dynamics and path planning, where understanding the interaction between the grapple, logs, terrain, and obstacles requires visual information. To address these challenges, we separate image segmentation from crane control and utilise a virtual camera to provide an image stream from reconstructed 3D data. We use Cartesian control to simplify domain transfer to real-world applications. Because log piles are static, visual servoing using a 3D reconstruction of the pile and its surroundings is equivalent to using real camera data until the point of grasping. This relaxes the limits on computational resources and time for the challenge of image segmentation, and allows for data collection in situations where the log piles are not occluded. The disadvantage is the lack of information during grasping. We demonstrate that this problem is manageable and present an agent that is 95% successful in picking one or several logs from challenging piles of 2–5 logs.

Place, publisher, year, edition, pages
MDPI, 2023
Keywords
autonomous forwarding, visual servoing, virtual camera, reinforcement learning, multi-log grasping, Cartesian control
National Category
Robotics Control Engineering Other Physics Topics Computer Vision and Robotics (Autonomous Systems)
Research subject
Physics; Automatic Control
Identifiers
urn:nbn:se:umu:diva-220402 (URN)10.3390/robotics13010003 (DOI)2-s2.0-85183350095 (Scopus ID)
Projects
Mistra Digital Forest
Funder
Mistra - The Swedish Foundation for Strategic Environmental Research, Grant DIA 2017/14 #6
Available from: 2024-02-02 Created: 2024-02-02 Last updated: 2024-02-09Bibliographically approved
Wiberg, V. (2023). Terrain machine learning. (Doctoral dissertation). Umeå: Umeå University
Open this publication in new window or tab >>Terrain machine learning
2023 (English)Doctoral thesis, comprehensive summary (Other academic)
Alternative title[sv]
Maskininlärning i terräng
Abstract [en]

The use of heavy vehicles in rough terrain is vital in the industry but has negative implications for the climate and ecosystem. In addition, the demand for improved efficiency underscores the need to enhance these vehicles' navigation capabilities. Navigating rough terrain presents distinct challenges, including deformable soil, surface roughness, and spatial and temporal terrain variability. Focusing on forestry, this thesis aims to improve navigation using machine learning and physics simulations. Without considering the vehicle-terrain dynamics, methods for navigation can result in unsafe or unnecessarily challenging situations. Specifically, we address route planning, control for autonomous vehicles, and soilde formations. We simulate soil using the discrete element method and vehicles using multibody dynamics.

To enhance route planning, we train a predictor model that uses a height map of the terrain to predict measures of traversability. The model has a directional dependency, couples geometric terrain features with vehicle design and dynamics, and allows for swift evaluations over large areas. The proposed method facilitates detailed route planning, using multiple objectives to yield efficient solutions.

We address autonomy in rough terrain navigation by training a controller through deep reinforcement learning. The control policy uses a local height map for perception to plan and control a forwarder with actively articulated suspensions. The controller adapts to overcome various obstacles and demonstrates skilled driving in rough terrain.

Extending beyond simulation, we address the simulation-to-reality gap of vehicles with complex hydraulic drivelines through system identification and domain randomization. The results show that having an accurate model of the actuators, modelling system delays, and preventing bang-bang control yields successful transfer. Controllers that train in simulation and transfer to reality are a step toward autonomous vehicles.

While the previously mentioned studies assume rigid terrain, we also answer if the discrete element method can capture large soil deformations due to heavy traffic. The results show that the discrete element method can represent a wide variety of natural soil and that the resulting rut depths agree well with empirical models and experimental data, including multipass scenarios.

Place, publisher, year, edition, pages
Umeå: Umeå University, 2023. p. 38
Keywords
multibody dynamics simulation, rough terrain vehicle, autonomous vehicles, robotics control, discrete element method, sim-to-real, reinforcement learning
National Category
Robotics Other Physics Topics Forest Science Computer Vision and Robotics (Autonomous Systems)
Research subject
Physics
Identifiers
urn:nbn:se:umu:diva-207982 (URN)978-91-8070-060-3 (ISBN)978-91-8070-059-7 (ISBN)
Public defence
2023-06-01, NAT.D.410, Umeå, 09:00 (English)
Opponent
Supervisors
Funder
Mistra - The Swedish Foundation for Strategic Environmental Research, DIA 2017/14 #6
Available from: 2023-05-11 Created: 2023-05-05 Last updated: 2023-05-08Bibliographically approved
Wiberg, V., Wallin, E., Nordjell, T. & Servin, M. (2022). Control of rough terrain vehicles using deep reinforcement learning. IEEE Robotics and Automation Letters, 7(1), 390-397
Open this publication in new window or tab >>Control of rough terrain vehicles using deep reinforcement learning
2022 (English)In: IEEE Robotics and Automation Letters, E-ISSN 2377-3766, Vol. 7, no 1, p. 390-397Article in journal (Refereed) Published
Abstract [en]

We explore the potential to control terrain vehicles using deep reinforcement in scenarios where human operators and traditional control methods are inadequate. This letter presents a controller that perceives, plans, and successfully controls a 16-tonne forestry vehicle with two frame articulation joints, six wheels, and their actively articulated suspensions to traverse rough terrain. The carefully shaped reward signal promotes safe, environmental, and efficient driving, which leads to the emergence of unprecedented driving skills. We test learned skills in a virtual environment, including terrains reconstructed from high-density laser scans of forest sites. The controller displays the ability to handle obstructing obstacles, slopes up to 27 degrees, and a variety of natural terrains, all with limited wheel slip, smooth, and upright traversal with intelligent use of the active suspensions. The results confirm that deep reinforcement learning has the potential to enhance control of vehicles with complex dynamics and high-dimensional observation data compared to human operators or traditional control methods, especially in rough terrain.

Place, publisher, year, edition, pages
IEEE, 2022
Keywords
Artificial Intelligence, Control and Optimization, Computer Science Applications, Computer Vision and Pattern Recognition, Mechanical Engineering, Human-Computer Interaction, Biomedical Engineering, Control and Systems Engineering
National Category
Robotics Computer Vision and Robotics (Autonomous Systems) Other Physics Topics
Research subject
Physics; Computer Science; Automatic Control
Identifiers
urn:nbn:se:umu:diva-189485 (URN)10.1109/lra.2021.3126904 (DOI)000721999500008 ()2-s2.0-85119416792 (Scopus ID)
Funder
Mistra - The Swedish Foundation for Strategic Environmental Research, 2017/14 #6eSSENCE - An eScience CollaborationSwedish National Infrastructure for Computing (SNIC), 2021/5-234
Available from: 2021-11-12 Created: 2021-11-12 Last updated: 2024-01-17Bibliographically approved
Wallin, E., Wiberg, V., Vesterlund, F., Holmgren, J., Persson, H. & Servin, M. (2022). Learning multiobjective rough terrain traversability. Journal of terramechanics, 102, 17-26
Open this publication in new window or tab >>Learning multiobjective rough terrain traversability
Show others...
2022 (English)In: Journal of terramechanics, ISSN 0022-4898, E-ISSN 1879-1204, Vol. 102, p. 17-26Article in journal (Refereed) Published
Abstract [en]

We present a method that uses high-resolution topography data of rough terrain, and ground vehicle simulation, to predict traversability. Traversability is expressed as three independent measures: the ability to traverse the terrain at a target speed, energy consumption, and acceleration. The measures are continuous and reflect different objectives for planning that go beyond binary classification. A deep neural network is trained to predict the traversability measures from the local heightmap and target speed. To produce training data, we use an articulated vehicle with wheeled bogie suspensions and procedurally generated terrains. We evaluate the model on laser-scanned forest terrains, previously unseen by the model. The model predicts traversability with an accuracy of 90%. Predictions rely on features from the high-dimensional terrain data that surpass local roughness and slope relative to the heading. Correlations show that the three traversability measures are complementary to each other. With an inference speed 3000 times faster than the ground truth simulation and trivially parallelizable, the model is well suited for traversability analysis and optimal path planning over large areas.

Place, publisher, year, edition, pages
Elsevier, 2022
Keywords
Traversability, Rough terrain vehicle, Multibody simulation, Laser scan, Deep learning
National Category
Robotics
Research subject
Physics; Computerized Image Analysis; data science; Physics
Identifiers
urn:nbn:se:umu:diva-193680 (URN)10.1016/j.jterra.2022.04.002 (DOI)000805039700001 ()2-s2.0-85129503568 (Scopus ID)
Projects
Mistra Digital Forest
Funder
Mistra - The Swedish Foundation for Strategic Environmental Research, 2017/14 #6eSSENCE - An eScience CollaborationSwedish National Infrastructure for Computing (SNIC), 2021/5-234
Available from: 2022-04-11 Created: 2022-04-11 Last updated: 2023-05-08Bibliographically approved
Wiberg, V., Wallin, E., Wadbro, E., Rosander, M. & Servin, M. (2022). Simulation-to-reality transfer to control a forwarder with active suspensions through deep reinforcement learning. In: International Conference of Forest Engineering COFE-FORMEC-IUFRO, Corvallis, Oregon, USA: . Paper presented at International Conference of Forest Engineering COFE-FORMEC-IUFRO, Corvallis, Oregon, USA, October 4-7, 2022.
Open this publication in new window or tab >>Simulation-to-reality transfer to control a forwarder with active suspensions through deep reinforcement learning
Show others...
2022 (English)In: International Conference of Forest Engineering COFE-FORMEC-IUFRO, Corvallis, Oregon, USA, 2022Conference paper, Oral presentation with published abstract (Other academic)
Abstract [en]

Automating the loaded and unloaded driving of a forwarder has the potential to reduce operational costsup to 10% in cut-to-length logging, but remains a challening and unsolved task. The complex interaction between the vehicle and terrain requires the controller to percieve its surroundings and thestate of the vehicle to plan for traversal. Because the state space is high dimensional and the systemdynamics cannot be formulated in closed form or easily approximated, traditional control methods areinadequate. Under these conditions, where learning to act in the environement is easier than learning thesystem dynamics, model free reinforcement learning is a promising option. We use deep reinforcementlearning for control of a 16-tonne forwarder with actively articulated suspensions. To efficiently gathergeneralizable experience, the control policies were safetly trained in simulation while varying severaldomain parameters. Each policy is trained during what correponds to roughly one month of real time.In simulation, the controller shows the ability to traverse rough terrains reconstruced from high-densitylaser scans and handles slopes up to 27◦. To compare the simulated to real performance we transfer thecontrol policies to the physical vehicle. Our results provide insight on how to improve policy transfer toheavy and expensive forest machines.

National Category
Robotics Computer Vision and Robotics (Autonomous Systems) Other Physics Topics
Research subject
Physics
Identifiers
urn:nbn:se:umu:diva-199447 (URN)
Conference
International Conference of Forest Engineering COFE-FORMEC-IUFRO, Corvallis, Oregon, USA, October 4-7, 2022
Available from: 2022-09-17 Created: 2022-09-17 Last updated: 2022-09-19
Lundbäck, M., Nordfjell, T., Wiberg, V., Wallin, E. & Servin, M. (2022). Traversability analysis using high-resolution laser-scans, simulation, and deep learning. In: : . Paper presented at International Conference of Forest Engineering COFE-FORMEC-IUFRO, Corvallis, Oregon, USA, October 4-7, 2022.
Open this publication in new window or tab >>Traversability analysis using high-resolution laser-scans, simulation, and deep learning
Show others...
2022 (English)Conference paper, Oral presentation with published abstract (Other academic)
Abstract [en]

Traversability is of major importance in forestry, where heavy vehicles, weighing up to 40 tons whenfully loaded, traverse rough and sometimes soft terrain. Forest remote sensing is becoming available atresolutions where surface roughness and slope can be determined at length-scales smaller than the forestmachines. Using 3D multibody dynamics simulation of a forest machine driving in virtual terrain replications, the interaction can be captured in great detail. The observed traversability is then automaticallya function of the vehicle geometry, dynamics, and of the local terrain topography relative to heading. Weexpress traversability with three complementary measures: i) the ability to traverse the terrain at a target speed, ii) energy consumption, and iii) machine body acceleration. For high traversability, the lattertwo should be as small as possible while the first measure is at maximum. The simulations are, however,too slow for systematically probing the traversability over large areas. Instead, a deep neural networkis trained to predict the traversability measures from the local heightmap and target speed. The trainingdata comes from simulations of an articulated vehicle with wheeled bogie suspensions driving over procedurally generated terrains while observing the dynamics and local terrain topology. We evaluate themodel on laser-scanned forest terrains, previously unseen by the model. The model predicts traversability with an accuracy of 90% on terrains with 0.25 m resolution and it is 3000 times faster than the groundtruth realtime simulation and trivially parallelizable, making it well suited for traversability analysis andoptimal route planning over large areas. The trained model depends on the vehicle heading, target speed,and detailed features in the topography that a model based only on local slope and roughness cannotcapture. We explore traversability statistics over large areas of laser-scanned terrains and discuss howthe model can be used as a complement or in place of the currently used terrain classification scheme.

National Category
Computer Vision and Robotics (Autonomous Systems) Robotics Other Physics Topics
Research subject
Physics
Identifiers
urn:nbn:se:umu:diva-199448 (URN)
Conference
International Conference of Forest Engineering COFE-FORMEC-IUFRO, Corvallis, Oregon, USA, October 4-7, 2022
Funder
Mistra - The Swedish Foundation for Strategic Environmental Research, 2017/14 #6
Available from: 2022-09-17 Created: 2022-09-17 Last updated: 2023-07-03
Wiberg, V., Servin, M. & Nordfjell, T. (2021). Discrete element modelling of large soil deformations under heavy vehicles. Journal of terramechanics, 93, 11-21
Open this publication in new window or tab >>Discrete element modelling of large soil deformations under heavy vehicles
2021 (English)In: Journal of terramechanics, ISSN 0022-4898, E-ISSN 1879-1204, Vol. 93, p. 11-21Article in journal (Refereed) Published
Abstract [en]

This paper addresses the challenges of creating realistic models of soil for simulations of heavy vehicles on weak terrain. We modelled dense soils using the discrete element method with variable parameters for surface friction, normal cohesion, and rolling resistance. To find out what type of soils can be represented, we measured the internal friction and bulk cohesion of over 100 different virtual samples. To test the model, we simulated rut formation from a heavy vehicle with different loads and soil strengths. We conclude that the relevant space of dense frictional and frictional-cohesive soils can be represented and that the model is applicable for simulation of large deformations induced by heavy vehicles on weak terrain.

Place, publisher, year, edition, pages
Elsevier, 2021
Keywords
DEM, Multibody Dynamics, Weak Soil, Rut Formation, Multipass
National Category
Other Physics Topics Other Earth and Related Environmental Sciences Applied Mechanics
Research subject
Physics
Identifiers
urn:nbn:se:umu:diva-176349 (URN)10.1016/j.jterra.2020.10.002 (DOI)000596712200002 ()2-s2.0-85094326100 (Scopus ID)
Funder
Mistra - The Swedish Foundation for Strategic Environmental Research, DIA 2017/14 #6eSSENCE - An eScience CollaborationSwedish National Infrastructure for Computing (SNIC), SNIC dnr 2019/3-168
Available from: 2020-11-01 Created: 2020-11-01 Last updated: 2023-05-08Bibliographically approved
Wiberg, V., Wallin, E., Fälldin, A., Semberg, T., Rossander, M., Wadbro, E. & Servin, M.Sim-to-real transfer of active suspension control using deep reinforcement learning.
Open this publication in new window or tab >>Sim-to-real transfer of active suspension control using deep reinforcement learning
Show others...
(English)Manuscript (preprint) (Other academic)
Abstract [en]

We explore sim-to-real transfer of deep reinforcement learning controllers for a heavy vehicle with active suspensions designed for traversing rough terrain. While related research primarily focuses on lightweight robots with electric motors and fast actuation, this study uses a forestry vehicle with a complex hydraulic driveline and slow actuation. We simulate the vehicle using multibody dynamics and apply system identification to find an appropriate set of simulation parameters. We then train policies in simulation using various techniques to mitigate the sim-to-real gap, including domain randomization, action delays, and a reward penalty to encourage smooth control. In reality, the policies trained with action delays and a penalty for erratic actions perform at nearly the same level as in simulation. In experiments on level ground, the motion trajectories closely overlap when turning to either side, as well as in a route tracking scenario. When faced with a ramp that requires active use of the suspensions, the simulated and real motions are in close alignment. This shows that the actuator model together with system identification yields a sufficiently accurate model of the actuators. We observe that policies trained without the additional action penalty exhibit fast switching or bang-bang control. These present smooth motions and high performance in simulation but transfer poorly to reality. We find that policies make marginal use of the local height map for perception, showing no indications of look-ahead planning. However, the strong transfer capabilities entail that further development concerning perception and performance can be largely confined to simulation. 

Keywords
autonomous vehicles, rough terrain navigation, machine learning, sim-to-real, reinforcement learning, heavy vehicles
National Category
Computer Vision and Robotics (Autonomous Systems) Other Physics Topics
Research subject
Physics; computer and systems sciences
Identifiers
urn:nbn:se:umu:diva-207977 (URN)10.48550/arXiv.2306.11171 (DOI)
Funder
Mistra - The Swedish Foundation for Strategic Environmental Research, DIA 2017/14 #6
Available from: 2023-05-05 Created: 2023-05-05 Last updated: 2023-07-04Bibliographically approved
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0001-6565-3123

Search in DiVA

Show all publications