Umeå universitets logga

umu.sePublikationer
Ändra sökning
Länk till posten
Permanent länk

Direktlänk
Publikationer (8 of 8) Visa alla publikationer
Wallin, E., Wiberg, V. & Servin, M. (2023). Multi-log grasping using reinforcement learning and virtual visual servoing. Robotics, 13(1), Article ID 3.
Öppna denna publikation i ny flik eller fönster >>Multi-log grasping using reinforcement learning and virtual visual servoing
2023 (Engelska)Ingår i: Robotics, E-ISSN 2218-6581, Vol. 13, nr 1, artikel-id 3Artikel i tidskrift (Refereegranskat) Published
Abstract [en]

We explore multi-log grasping using reinforcement learning and virtual visual servoing for automated forwarding in a simulated environment. Automation of forest processes is a major challenge, and many techniques regarding robot control pose different challenges due to the unstructured and harsh outdoor environment. Grasping multiple logs involves various problems of dynamics and path planning, where understanding the interaction between the grapple, logs, terrain, and obstacles requires visual information. To address these challenges, we separate image segmentation from crane control and utilise a virtual camera to provide an image stream from reconstructed 3D data. We use Cartesian control to simplify domain transfer to real-world applications. Because log piles are static, visual servoing using a 3D reconstruction of the pile and its surroundings is equivalent to using real camera data until the point of grasping. This relaxes the limits on computational resources and time for the challenge of image segmentation, and allows for data collection in situations where the log piles are not occluded. The disadvantage is the lack of information during grasping. We demonstrate that this problem is manageable and present an agent that is 95% successful in picking one or several logs from challenging piles of 2–5 logs.

Ort, förlag, år, upplaga, sidor
MDPI, 2023
Nyckelord
autonomous forwarding, visual servoing, virtual camera, reinforcement learning, multi-log grasping, Cartesian control
Nationell ämneskategori
Robotteknik och automation Reglerteknik Annan fysik Datorseende och robotik (autonoma system)
Forskningsämne
fysik; reglerteknik
Identifikatorer
urn:nbn:se:umu:diva-220402 (URN)10.3390/robotics13010003 (DOI)2-s2.0-85183350095 (Scopus ID)
Projekt
Mistra Digital Forest
Forskningsfinansiär
Mistra - Stiftelsen för miljöstrategisk forskning, Grant DIA 2017/14 #6
Tillgänglig från: 2024-02-02 Skapad: 2024-02-02 Senast uppdaterad: 2024-02-09Bibliografiskt granskad
Wiberg, V. (2023). Terrain machine learning. (Doctoral dissertation). Umeå: Umeå University
Öppna denna publikation i ny flik eller fönster >>Terrain machine learning
2023 (Engelska)Doktorsavhandling, sammanläggning (Övrigt vetenskapligt)
Alternativ titel[sv]
Maskininlärning i terräng
Abstract [en]

The use of heavy vehicles in rough terrain is vital in the industry but has negative implications for the climate and ecosystem. In addition, the demand for improved efficiency underscores the need to enhance these vehicles' navigation capabilities. Navigating rough terrain presents distinct challenges, including deformable soil, surface roughness, and spatial and temporal terrain variability. Focusing on forestry, this thesis aims to improve navigation using machine learning and physics simulations. Without considering the vehicle-terrain dynamics, methods for navigation can result in unsafe or unnecessarily challenging situations. Specifically, we address route planning, control for autonomous vehicles, and soilde formations. We simulate soil using the discrete element method and vehicles using multibody dynamics.

To enhance route planning, we train a predictor model that uses a height map of the terrain to predict measures of traversability. The model has a directional dependency, couples geometric terrain features with vehicle design and dynamics, and allows for swift evaluations over large areas. The proposed method facilitates detailed route planning, using multiple objectives to yield efficient solutions.

We address autonomy in rough terrain navigation by training a controller through deep reinforcement learning. The control policy uses a local height map for perception to plan and control a forwarder with actively articulated suspensions. The controller adapts to overcome various obstacles and demonstrates skilled driving in rough terrain.

Extending beyond simulation, we address the simulation-to-reality gap of vehicles with complex hydraulic drivelines through system identification and domain randomization. The results show that having an accurate model of the actuators, modelling system delays, and preventing bang-bang control yields successful transfer. Controllers that train in simulation and transfer to reality are a step toward autonomous vehicles.

While the previously mentioned studies assume rigid terrain, we also answer if the discrete element method can capture large soil deformations due to heavy traffic. The results show that the discrete element method can represent a wide variety of natural soil and that the resulting rut depths agree well with empirical models and experimental data, including multipass scenarios.

Ort, förlag, år, upplaga, sidor
Umeå: Umeå University, 2023. s. 38
Nyckelord
multibody dynamics simulation, rough terrain vehicle, autonomous vehicles, robotics control, discrete element method, sim-to-real, reinforcement learning
Nationell ämneskategori
Robotteknik och automation Annan fysik Skogsvetenskap Datorseende och robotik (autonoma system)
Forskningsämne
fysik
Identifikatorer
urn:nbn:se:umu:diva-207982 (URN)978-91-8070-060-3 (ISBN)978-91-8070-059-7 (ISBN)
Disputation
2023-06-01, NAT.D.410, Umeå, 09:00 (Engelska)
Opponent
Handledare
Forskningsfinansiär
Mistra - Stiftelsen för miljöstrategisk forskning, DIA 2017/14 #6
Tillgänglig från: 2023-05-11 Skapad: 2023-05-05 Senast uppdaterad: 2023-05-08Bibliografiskt granskad
Wiberg, V., Wallin, E., Nordjell, T. & Servin, M. (2022). Control of rough terrain vehicles using deep reinforcement learning. IEEE Robotics and Automation Letters, 7(1), 390-397
Öppna denna publikation i ny flik eller fönster >>Control of rough terrain vehicles using deep reinforcement learning
2022 (Engelska)Ingår i: IEEE Robotics and Automation Letters, E-ISSN 2377-3766, Vol. 7, nr 1, s. 390-397Artikel i tidskrift (Refereegranskat) Published
Abstract [en]

We explore the potential to control terrain vehicles using deep reinforcement in scenarios where human operators and traditional control methods are inadequate. This letter presents a controller that perceives, plans, and successfully controls a 16-tonne forestry vehicle with two frame articulation joints, six wheels, and their actively articulated suspensions to traverse rough terrain. The carefully shaped reward signal promotes safe, environmental, and efficient driving, which leads to the emergence of unprecedented driving skills. We test learned skills in a virtual environment, including terrains reconstructed from high-density laser scans of forest sites. The controller displays the ability to handle obstructing obstacles, slopes up to 27 degrees, and a variety of natural terrains, all with limited wheel slip, smooth, and upright traversal with intelligent use of the active suspensions. The results confirm that deep reinforcement learning has the potential to enhance control of vehicles with complex dynamics and high-dimensional observation data compared to human operators or traditional control methods, especially in rough terrain.

Ort, förlag, år, upplaga, sidor
IEEE, 2022
Nyckelord
Artificial Intelligence, Control and Optimization, Computer Science Applications, Computer Vision and Pattern Recognition, Mechanical Engineering, Human-Computer Interaction, Biomedical Engineering, Control and Systems Engineering
Nationell ämneskategori
Robotteknik och automation Datorseende och robotik (autonoma system) Annan fysik
Forskningsämne
fysik; datalogi; reglerteknik
Identifikatorer
urn:nbn:se:umu:diva-189485 (URN)10.1109/lra.2021.3126904 (DOI)000721999500008 ()2-s2.0-85119416792 (Scopus ID)
Forskningsfinansiär
Mistra - Stiftelsen för miljöstrategisk forskning, 2017/14 #6eSSENCE - An eScience CollaborationSwedish National Infrastructure for Computing (SNIC), 2021/5-234
Tillgänglig från: 2021-11-12 Skapad: 2021-11-12 Senast uppdaterad: 2024-01-17Bibliografiskt granskad
Wallin, E., Wiberg, V., Vesterlund, F., Holmgren, J., Persson, H. & Servin, M. (2022). Learning multiobjective rough terrain traversability. Journal of terramechanics, 102, 17-26
Öppna denna publikation i ny flik eller fönster >>Learning multiobjective rough terrain traversability
Visa övriga...
2022 (Engelska)Ingår i: Journal of terramechanics, ISSN 0022-4898, E-ISSN 1879-1204, Vol. 102, s. 17-26Artikel i tidskrift (Refereegranskat) Published
Abstract [en]

We present a method that uses high-resolution topography data of rough terrain, and ground vehicle simulation, to predict traversability. Traversability is expressed as three independent measures: the ability to traverse the terrain at a target speed, energy consumption, and acceleration. The measures are continuous and reflect different objectives for planning that go beyond binary classification. A deep neural network is trained to predict the traversability measures from the local heightmap and target speed. To produce training data, we use an articulated vehicle with wheeled bogie suspensions and procedurally generated terrains. We evaluate the model on laser-scanned forest terrains, previously unseen by the model. The model predicts traversability with an accuracy of 90%. Predictions rely on features from the high-dimensional terrain data that surpass local roughness and slope relative to the heading. Correlations show that the three traversability measures are complementary to each other. With an inference speed 3000 times faster than the ground truth simulation and trivially parallelizable, the model is well suited for traversability analysis and optimal path planning over large areas.

Ort, förlag, år, upplaga, sidor
Elsevier, 2022
Nyckelord
Traversability, Rough terrain vehicle, Multibody simulation, Laser scan, Deep learning
Nationell ämneskategori
Robotteknik och automation
Forskningsämne
fysik; datoriserad bildanalys; data science; fysik
Identifikatorer
urn:nbn:se:umu:diva-193680 (URN)10.1016/j.jterra.2022.04.002 (DOI)000805039700001 ()2-s2.0-85129503568 (Scopus ID)
Projekt
Mistra Digital Forest
Forskningsfinansiär
Mistra - Stiftelsen för miljöstrategisk forskning, 2017/14 #6eSSENCE - An eScience CollaborationSwedish National Infrastructure for Computing (SNIC), 2021/5-234
Tillgänglig från: 2022-04-11 Skapad: 2022-04-11 Senast uppdaterad: 2023-05-08Bibliografiskt granskad
Wiberg, V., Wallin, E., Wadbro, E., Rosander, M. & Servin, M. (2022). Simulation-to-reality transfer to control a forwarder with active suspensions through deep reinforcement learning. In: International Conference of Forest Engineering COFE-FORMEC-IUFRO, Corvallis, Oregon, USA: . Paper presented at International Conference of Forest Engineering COFE-FORMEC-IUFRO, Corvallis, Oregon, USA, October 4-7, 2022.
Öppna denna publikation i ny flik eller fönster >>Simulation-to-reality transfer to control a forwarder with active suspensions through deep reinforcement learning
Visa övriga...
2022 (Engelska)Ingår i: International Conference of Forest Engineering COFE-FORMEC-IUFRO, Corvallis, Oregon, USA, 2022Konferensbidrag, Muntlig presentation med publicerat abstract (Övrigt vetenskapligt)
Abstract [en]

Automating the loaded and unloaded driving of a forwarder has the potential to reduce operational costsup to 10% in cut-to-length logging, but remains a challening and unsolved task. The complex interaction between the vehicle and terrain requires the controller to percieve its surroundings and thestate of the vehicle to plan for traversal. Because the state space is high dimensional and the systemdynamics cannot be formulated in closed form or easily approximated, traditional control methods areinadequate. Under these conditions, where learning to act in the environement is easier than learning thesystem dynamics, model free reinforcement learning is a promising option. We use deep reinforcementlearning for control of a 16-tonne forwarder with actively articulated suspensions. To efficiently gathergeneralizable experience, the control policies were safetly trained in simulation while varying severaldomain parameters. Each policy is trained during what correponds to roughly one month of real time.In simulation, the controller shows the ability to traverse rough terrains reconstruced from high-densitylaser scans and handles slopes up to 27◦. To compare the simulated to real performance we transfer thecontrol policies to the physical vehicle. Our results provide insight on how to improve policy transfer toheavy and expensive forest machines.

Nationell ämneskategori
Robotteknik och automation Datorseende och robotik (autonoma system) Annan fysik
Forskningsämne
fysik
Identifikatorer
urn:nbn:se:umu:diva-199447 (URN)
Konferens
International Conference of Forest Engineering COFE-FORMEC-IUFRO, Corvallis, Oregon, USA, October 4-7, 2022
Tillgänglig från: 2022-09-17 Skapad: 2022-09-17 Senast uppdaterad: 2022-09-19
Lundbäck, M., Nordfjell, T., Wiberg, V., Wallin, E. & Servin, M. (2022). Traversability analysis using high-resolution laser-scans, simulation, and deep learning. In: : . Paper presented at International Conference of Forest Engineering COFE-FORMEC-IUFRO, Corvallis, Oregon, USA, October 4-7, 2022.
Öppna denna publikation i ny flik eller fönster >>Traversability analysis using high-resolution laser-scans, simulation, and deep learning
Visa övriga...
2022 (Engelska)Konferensbidrag, Muntlig presentation med publicerat abstract (Övrigt vetenskapligt)
Abstract [en]

Traversability is of major importance in forestry, where heavy vehicles, weighing up to 40 tons whenfully loaded, traverse rough and sometimes soft terrain. Forest remote sensing is becoming available atresolutions where surface roughness and slope can be determined at length-scales smaller than the forestmachines. Using 3D multibody dynamics simulation of a forest machine driving in virtual terrain replications, the interaction can be captured in great detail. The observed traversability is then automaticallya function of the vehicle geometry, dynamics, and of the local terrain topography relative to heading. Weexpress traversability with three complementary measures: i) the ability to traverse the terrain at a target speed, ii) energy consumption, and iii) machine body acceleration. For high traversability, the lattertwo should be as small as possible while the first measure is at maximum. The simulations are, however,too slow for systematically probing the traversability over large areas. Instead, a deep neural networkis trained to predict the traversability measures from the local heightmap and target speed. The trainingdata comes from simulations of an articulated vehicle with wheeled bogie suspensions driving over procedurally generated terrains while observing the dynamics and local terrain topology. We evaluate themodel on laser-scanned forest terrains, previously unseen by the model. The model predicts traversability with an accuracy of 90% on terrains with 0.25 m resolution and it is 3000 times faster than the groundtruth realtime simulation and trivially parallelizable, making it well suited for traversability analysis andoptimal route planning over large areas. The trained model depends on the vehicle heading, target speed,and detailed features in the topography that a model based only on local slope and roughness cannotcapture. We explore traversability statistics over large areas of laser-scanned terrains and discuss howthe model can be used as a complement or in place of the currently used terrain classification scheme.

Nationell ämneskategori
Datorseende och robotik (autonoma system) Robotteknik och automation Annan fysik
Forskningsämne
fysik
Identifikatorer
urn:nbn:se:umu:diva-199448 (URN)
Konferens
International Conference of Forest Engineering COFE-FORMEC-IUFRO, Corvallis, Oregon, USA, October 4-7, 2022
Forskningsfinansiär
Mistra - Stiftelsen för miljöstrategisk forskning, 2017/14 #6
Tillgänglig från: 2022-09-17 Skapad: 2022-09-17 Senast uppdaterad: 2023-07-03
Wiberg, V., Servin, M. & Nordfjell, T. (2021). Discrete element modelling of large soil deformations under heavy vehicles. Journal of terramechanics, 93, 11-21
Öppna denna publikation i ny flik eller fönster >>Discrete element modelling of large soil deformations under heavy vehicles
2021 (Engelska)Ingår i: Journal of terramechanics, ISSN 0022-4898, E-ISSN 1879-1204, Vol. 93, s. 11-21Artikel i tidskrift (Refereegranskat) Published
Abstract [en]

This paper addresses the challenges of creating realistic models of soil for simulations of heavy vehicles on weak terrain. We modelled dense soils using the discrete element method with variable parameters for surface friction, normal cohesion, and rolling resistance. To find out what type of soils can be represented, we measured the internal friction and bulk cohesion of over 100 different virtual samples. To test the model, we simulated rut formation from a heavy vehicle with different loads and soil strengths. We conclude that the relevant space of dense frictional and frictional-cohesive soils can be represented and that the model is applicable for simulation of large deformations induced by heavy vehicles on weak terrain.

Ort, förlag, år, upplaga, sidor
Elsevier, 2021
Nyckelord
DEM, Multibody Dynamics, Weak Soil, Rut Formation, Multipass
Nationell ämneskategori
Annan fysik Annan geovetenskap och miljövetenskap Teknisk mekanik
Forskningsämne
fysik
Identifikatorer
urn:nbn:se:umu:diva-176349 (URN)10.1016/j.jterra.2020.10.002 (DOI)000596712200002 ()2-s2.0-85094326100 (Scopus ID)
Forskningsfinansiär
Mistra - Stiftelsen för miljöstrategisk forskning, DIA 2017/14 #6eSSENCE - An eScience CollaborationSwedish National Infrastructure for Computing (SNIC), SNIC dnr 2019/3-168
Tillgänglig från: 2020-11-01 Skapad: 2020-11-01 Senast uppdaterad: 2023-05-08Bibliografiskt granskad
Wiberg, V., Wallin, E., Fälldin, A., Semberg, T., Rossander, M., Wadbro, E. & Servin, M.Sim-to-real transfer of active suspension control using deep reinforcement learning.
Öppna denna publikation i ny flik eller fönster >>Sim-to-real transfer of active suspension control using deep reinforcement learning
Visa övriga...
(Engelska)Manuskript (preprint) (Övrigt vetenskapligt)
Abstract [en]

We explore sim-to-real transfer of deep reinforcement learning controllers for a heavy vehicle with active suspensions designed for traversing rough terrain. While related research primarily focuses on lightweight robots with electric motors and fast actuation, this study uses a forestry vehicle with a complex hydraulic driveline and slow actuation. We simulate the vehicle using multibody dynamics and apply system identification to find an appropriate set of simulation parameters. We then train policies in simulation using various techniques to mitigate the sim-to-real gap, including domain randomization, action delays, and a reward penalty to encourage smooth control. In reality, the policies trained with action delays and a penalty for erratic actions perform at nearly the same level as in simulation. In experiments on level ground, the motion trajectories closely overlap when turning to either side, as well as in a route tracking scenario. When faced with a ramp that requires active use of the suspensions, the simulated and real motions are in close alignment. This shows that the actuator model together with system identification yields a sufficiently accurate model of the actuators. We observe that policies trained without the additional action penalty exhibit fast switching or bang-bang control. These present smooth motions and high performance in simulation but transfer poorly to reality. We find that policies make marginal use of the local height map for perception, showing no indications of look-ahead planning. However, the strong transfer capabilities entail that further development concerning perception and performance can be largely confined to simulation. 

Nyckelord
autonomous vehicles, rough terrain navigation, machine learning, sim-to-real, reinforcement learning, heavy vehicles
Nationell ämneskategori
Datorseende och robotik (autonoma system) Annan fysik
Forskningsämne
fysik; data- och systemvetenskap
Identifikatorer
urn:nbn:se:umu:diva-207977 (URN)10.48550/arXiv.2306.11171 (DOI)
Forskningsfinansiär
Mistra - Stiftelsen för miljöstrategisk forskning, DIA 2017/14 #6
Tillgänglig från: 2023-05-05 Skapad: 2023-05-05 Senast uppdaterad: 2023-07-04Bibliografiskt granskad
Organisationer
Identifikatorer
ORCID-id: ORCID iD iconorcid.org/0000-0001-6565-3123

Sök vidare i DiVA

Visa alla publikationer