Umeå University's logo

The problem of generating oscillations of the inertia wheel pendulum is considered. We combine exact feedback linearization with two-relay controller, tuned using frequency-domain tools, such as computing the locus of a perturbed relay system. Explicit expressions for the parameters of the controller in terms of the desired frequency and amplitude are derived. Sufficient conditions for orbital asymptotic stability of the closed-loop system are obtained with the help of the Poincare map. Performance is validated via experiments. The approach can be easily applied for a minimum phase system, provided the behavior of the states of the zero dynamics is of no concern. Copyright (C) 2011 John Wiley & Sons, Ltd.

The purpose of the paper is to analyze the performance of several global position regulators for robot manipulators with Coulomb friction. All the controllers include a proportional-differential part and a switched part whereas the difference between the controllers is in the way of compensation of the gravitational forces. Stability analysis is also revisited within the nonsmooth Lyapunov function framework for the controllers with and without gravity pre-compensation. Performance issues of the proposed controllers are evaluated in an experimental study of a five degrees-of-freedom robot manipulator. In the experiments, we choose two criteria for performance analysis. In the first set of experiments, we set the same gains to all the controllers. In the second set of experiments, the gains of the controller were chosen such that the work done by the manipulator is similar.

Forestry machines are heavy vehicles performing complex manipulation tasks in unstructured production forest environments. Together with the complex dynamics of the on-board hydraulically actuated cranes, the rough forest terrains have posed a particular challenge in forestry automation. In this study, the feasibility of applying reinforcement learning control to forestry crane manipulators is investigated in a simulated environment. Our results show that it is possible to learn successful actuator-space control policies for energy efficient log grasping by invoking a simple curriculum in a deep reinforcement learning setup. Given the pose of the selected logs, our best control policy reaches a grasping success rate of 97%. Including an energy-optimization goal in the reward function, the energy consumption is significantly reduced compared to control policies learned without incentive for energy optimization, while the increase in cycle time is marginal. The energy-optimization effects can be observed in the overall smoother motion and acceleration profiles during crane manipulation. 

This paper presents a method using data-driven models for selecting actions and predicting the total performance of autonomous wheel loader operations over many loading cycles in a changing environment. The performance includes loaded mass, loading time, work. The data-driven models input the control parameters of a loading action and the heightmap of the initial pile state to output the inference of either the performance or the resulting pile state. By iteratively utilizing the resulting pile state as the initial pile state for consecutive predictions, the prediction method enables long-horizon forecasting. Deep neural networks were trained on data from over 10,000 random loading actions in gravel piles of different shapes using 3D multibody dynamics simulation. The models predict the performance and the resulting pile state with, on average, 95% accuracy in 1.2 ms, and 97% in 4.5 ms, respectively. The performance prediction was found to be even faster in exchange for accuracy by reducing the model size with the lower dimensional representation of the pile state using its slope and curvature. The feasibility of long-horizon predictions was confirmed with 40 sequential loading actions at a large pile. With the aid of a physics-based model, the pile state predictions are kept sufficiently accurate for longer-horizon use.

This paper presents a method for learning world models for wheel loaders performing automatic loading actions on a pile of soil. Data-driven models were learned to output the resulting pile state, loaded mass, time, and work for a single loading cycle given inputs that include a heightmap of the initial pile shape and action parameters for an automatic bucket-filling controller. Long-horizon planning of sequential loading in a dynamically changing environment is thus enabled as repeated model inference. The models, consisting of deep neural networks, were trained on data from a 3D multibody dynamics simulation of over 10,000 random loading actions in gravel piles of different shapes. The accuracy and inference time for predicting the loading performance and the resulting pile state were, on average, 95% in 1.21.2 ms and 97% in 4.54.5 ms, respectively. Long-horizon predictions were found feasible over 40 sequential loading actions.

We investigate how well a physics-based simulator can replicate a real wheel loader performing bucket filling in a pile of soil. The comparison is made using field-test time series of the vehicle motion and actuation forces, loaded mass, and total work. The vehicle was modeled as a rigid multibody system with frictional contacts, driveline, and linear actuators. For the soil, we tested discrete-element models of different resolutions, with and without multiscale acceleration. The spatiotemporal resolution ranged between 50–400 mm and 2–500 ms, and the computational speed was between 1/10,000 to 5 times faster than real time. The simulation-to-reality gap was found to be around 10% and exhibited a weak dependence on the level of fidelity, e.g., compatible with real-time simulation. Furthermore, the sensitivity of an optimized force-feedback controller under transfer between different simulation domains was investigated. The domain bias was observed to cause a performance reduction of 5% despite the domain gap being about 15%.

Having smart and autonomous earthmoving in mind, we explore high-performance wheel loading in a simulated environment. This paper introduces a wheel loader simulator that combines contacting 3D multibody dynamics with a hybrid continuum-particle terrain model, supporting realistic digging forces and soil displacements at real-time performance. A total of 270,000 simulations are run with different loading actions, pile slopes, and soil to analyze how they affect the loading performance. The results suggest that the preferred digging actions should preserve and exploit a steep pile slope. High digging speed favors high productivity, while energy-efficient loading requires a lower dig speed. 

Framstegen inom AI väcker frågor kring den militära tilllämpningen av tekniken och tillåten grad av automatisering av vapensystem. Debatten om AI i militära tillämpningar förs både av civilsamhällets organisationer och av stater. Debatten började med det mycket specifika ifrågasättandet av fullt autonoma vapensystem, ofta kallade Lethal Autonomous Weapon Systems (LAWS), men debatten och ifrågasättandet har vidgats till att omfatta införandet av AI i militära tillämpningar. Diskussionen är mångfacetterad och präglas av flera olika perspektiv på ställningstaganden och argumentation. Autonoma system har många användningsområden i militära tillämpningar och det är viktigt att förstå hur integrationen av högautomatiserade system ska kunna göras medavseende på de rättsliga ramverk som försvaret lyder under.

Även om debatten till viss del har sitt ursprung i det senaste decenniets drönarkrigföring är de flesta parter överens omatt dagens debatt snarare handlar om framtida teknik. De tekniska, militära och juridiska aspekterna är centrala i diskussionen men även etiska, psykologiska och säkerhetspolitiska aspekter tar plats i debatten.

This paper presents the development, testing and validation of SWEEPER, a robot for harvesting sweet pepper fruit in greenhouses. The robotic system includes a six degrees of freedom industrial arm equipped with a specially designed end effector, RGB-D camera, high-end computer with graphics processing unit, programmable logic controllers, other electronic equipment, and a small container to store harvested fruit. All is mounted on a cart that autonomously drives on pipe rails and concrete floor in the end-user environment. The overall operation of the harvesting robot is described along with details of the algorithms for fruit detection and localization, grasp pose estimation, and motion control. The main contributions of this paper are the integrated system design and its validation and extensive field testing in a commercial greenhouse for different varieties and growing conditions. A total of 262 fruits were involved in a 4-week long testing period. The average cycle time to harvest a fruit was 24 s. Logistics took approximately 50% of this time (7.8 s for discharge of fruit and 4.7 s for platform movements). Laboratory experiments have proven that the cycle time can be reduced to 15 s by running the robot manipulator at a higher speed. The harvest success rates were 61% for the best fit crop conditions and 18% in current crop conditions. This reveals the importance of finding the best fit crop conditions and crop varieties for successful robotic harvesting. The SWEEPER robot is the first sweet pepper harvesting robot to demonstrate this kind of performance in a commercial greenhouse.

Machine vision systems combined with classification algorithms are being increasingly used for different applications in the age of automation. One such application would be the quality control of the painted automobile parts. The fundamental elements of the machine vision system include camera, illumination, image acquisition software and computer vision algorithms. Traditional way of thinking puts too much importance on camera systems and ignores other elements while designing a machine vision system. In this thesis work, it is shown that selecting an appropriate illumination for illuminating the surface being examined is equally important in case of machine vision system for examining specular surface. Knowledge about the nature of the surface, type and properties of the defect to be detected and classified are important factors while choosing the illumination system for the machine vision system. The main illumination system tested were bright field, dark field and structured illumination and out of the three, dark field and structured illumination gave best results.

This thesis work proposes a dark field illumination based machine vision system for fault detection on specular painted surface. A single layer Artificial Neural Network model is employed for the classification of defects in intensity images of painted surface acquired with this machine vision system. The results of this research work proved that the quality of the images and size of data set used for training the Neural Network model play a vital role in the performance of the classifier algorithm.

Brain-computer interface (BCI) systems can provide people with ability to communicate and control real world systems using neural activities. Therefore, it makes sense to develop an assistive framework for command and control of a future robotic system which can assist the human robot collaboration. In this paper, we have employed electroencephalographic (EEG) signals recorded by electrodes placed over the scalp. The human-hand movement based motor imagery mentalization is used to collect brain signals over the motor cortex area. The collected µ-wave (8–13 Hz) EEG signals were analyzed with event-related desynchronization/synchronization (ERD/ERS) quantification to extract a threshold between hand grip and release movement and this information can be used to control forestry crane grasping and release functionality. The experiment was performed with four healthy persons to demonstrate the proof-of concept BCI system. From this study, it is demonstrated that the proposed method has potential to assist the manual operation of crane operators performing advanced task with heavy cognitive work load.

The reinforcement learning control of an underground loader was investigated in a simulated environment by using a multi-agent deep neural network approach. At the start of each loading cycle, one agent selects the dig position from a depth camera image of a pile of fragmented rock. A second agent is responsible for continuous control of the vehicle, with the goal of filling the bucket at the selected loading point while avoiding collisions, getting stuck, or losing ground traction. This relies on motion and force sensors, as well as on a camera and lidar. Using a soft actor–critic algorithm, the agents learn policies for efficient bucket filling over many subsequent loading cycles, with a clear ability to adapt to the changing environment. The best results—on average, 75% of the max capacity—were obtained when including a penalty for energy usage in the reward.

Unmanned aerial systems have been widely used for variety of civilian applications over the past few years. Some of these applications require accurate guidance and control. Consequently, Unmanned Aerial Vehicle (UAV) guidance and control attracted many researchers in both control theory and aerospace engineering. Flying wings, as a particular type of UAV, are considered to have one of the most efficient aerodynamic structures. It is however difficult to design robust controller for such systems. This is due to the fact that flying wings are highly sensitive to control inputs.

The focus of this thesis is on modeling and control design for a UAV system. The platform understudy is a flying wing developed by SmartPlanes Co. located in Skellefteå, Sweden. This UAV is particularly used for topological mapping and aerial photography.

The novel approach suggested in this thesis is to use two controllers in sequence. More precisely, Linear Quadratic Regulator (LQR) is suggested to provide robust stability, and Proportional, Integral, Derivative (PID) controller is suggested to provide reference signal regulation. The idea behind this approach is that with LQR in the loop, the system becomes more stable and less sensitive to control signals. Thus, PID controller has an easier task to do, and is only used to provide the required transient response.

The closed-loop system containing the developed controller and a UAV non-linear dynamic model was simulated in Simulink. Simulated controller was then tested for stability and robustness with respect to some parametric uncertainty. Obtained results revealed that the LQR successfully managed to provide robust stability, and PID provided reference signal regulation.

Federated Learning (FL) has evolved as a promising distributed learning paradigm in which data samples are disseminated over massively connected devices in an IID (Identical and Independent Distribution) or non-IID manner. FL follows a collaborative training approach where each device uses local training data to train local models, and the server generates a global model by combining the local model's parameters. However, FL is vulnerable to system heterogeneity when local devices have varying computational, storage, and communication capabilities over time. The presence of stragglers or low-performing devices in the learning process severely impacts the scalability of FL algorithms and significantly delays convergence. To mitigate this problem, we propose Fed-MOODS, a Multi-Objective Optimization-based Device Selection approach to reduce the effect of stragglers in the FL process. The primary criteria for optimization are to maximize: (i) the availability of the processing capacity of each device, (ii) the availability of the memory in devices, and (iii) the bandwidth capacity of the participating devices. The multi-objective optimization prioritizes devices from fast to slow. The approach involves faster devices in early global rounds and gradually incorporating slower devices from the Pareto fronts to improve the model's accuracy. The overall training time of Fed-MOODS is 1.8× and 1.48× faster than the baseline model (FedAvg) with random device selection for MNIST and FMNIST non-IID data, respectively. Fed-MOODS is extensively evaluated under multiple experimental settings, and the results show that Fed-MOODS has significantly improved model's convergence and performance. Fed-MOODS maintains fairness in the prioritized participation of devices and the model for both IID and non-IID settings.

In verbal human-robot interaction natural language utterances have to be grounded in visual scenes by the robot. Visual language grounding is a challenging task that includes identifying a primary object among several objects, together with the object properties and spatial relations among the objects. In this paper we focus on extracting this information from sentences only. We propose two language modelling techniques, one uses regular expressions and the other one utilizes Euclidian distance. We compare these two proposed techniques with two other techniques that utilize tree structures, namely an extended Hobb’s algorithm and an algorithm that utilizes a Stanford parse tree. A comparative analysis between all language modelling techniques shows that our proposed two approaches require less computational time than the tree-based approaches. All approaches perform good identifying the primary object and its property, but for spatial relation extraction the Stanford parse tree algorithm performs better than the other language modelling techniques. Time elapsed for the Stanford parse tree algorithm is higher than for the other techniques.

An approach for decision-level fusion for gesture and speech based human-robot interaction (HRI) is proposed. A rule-based method is compared with several machine learning approaches. Gestures and speech signals are initially classified using hidden Markov models, reaching accuracies of 89.6% and 84% respectively. The rule-based approach reached 91.6% while SVM, which was the best of all evaluated machine learning algorithms, reached an accuracy of 98.2% on the test data. A complete framework is deployed in real time humanoid robot (NAO) which proves the efficacy of the system.

We report on experiences of using the ROS middleware for developmentof agricultural robots. We describe software related design considerations for all maincomponents in developed subsystems as well as drawbacks and advantages with thechosen approaches. This work was partly funded by the European Commission(CROPS GA no 246252).

In many complex robotics systems, interaction takes place in all directions between human, robot, and environment. Performance of such a system depends on this interaction, and a proper evaluation of a system must build on a proper modeling of interaction, a relevant set of performance metrics, and a methodology to combine metrics into a single performance value. In this paper, existing models of human-robot interaction are adapted to fit complex scenarios with one or several humans and robots. The interaction and the evaluation process is formalized, and a general method to fuse performance values over time and for several performance metrics is presented. The resulting value, denoted interaction quality, adds a dimension to ordinary performance metrics by being explicit about the interplay between performance metrics, and thereby provides a formal framework to understand, model, and address complex aspects of evaluation of human-robot interaction. 

One important aspect when designing understandable robots is how robots should communicate with a human user to be understood in the best way. In elder care applications this is particularly important, and also difficult since many older adults suffer from various kinds of impairments. In this paper we present a solution where communication modality and communication parameters are adapted to fit both a user profile and an environment model comprising information about light and sound conditions that may affect communication. The Rasa dialogue manager is complemented with necessary functionality, and the operation is verified with a Pepper robot interacting with several personas with impaired vision, hearing, and cognition. Several relevant ethical questions are identified and briefly discussed, as a contribution to the WARN workshop.

A lot of features that need to be scanned underground should not be disturbed, from waterlines to unmarked graves. A non-invasive way of probing underground is Ground-Penetrating Radar (GPR). GPR finds differences in materials with radar waves. However, GPR is human-operated and its position is generally determined with a GPS. In some cases, the presence of a human operator can be dangerous, and in other cases, the GPS is not reliable (i.e. mines, glaciers). Therefore there are situations where an autonomous and non-GPS-reliant solution is preferable. The current state of the autonomous GPR system targeted in this work has a non-intuitive GUI that requires an experienced hand to operate. I present an updated hardware and software platform with an intuitive GUI. This updated autonomous system continuously builds a map of its surroundings with Simultaneous Mapping And Localization (SLAM). SLAM localizes itself within the map through sensor-fused position estimates. After the survey is completed the positions are saved and integrated with radar data to be visualized. Robot Operating System 2 (ROS2) is the software I used that allows communications between hardware components, software systems, and the GUI. The new hardware package uses only one source of power and is built using quick connectors that allow for quick removal from the GPR platform. This system allows for intuitive autonomous survey planning and execution in any field paired with a simple way of visualizing data.

A method for Learning from Demonstration (LFD) is presented and evaluated on a simulated Robosoft Kompai robot. The presented algorithm, called Predictive Sequence Learning (PSL), builds fuzzy rules describing temporal relations between sensory-motor events recorded while a human operator is tele-operating the robot. The generated rule base can be used to control the robot and to predict expected sensor events in response to executed actions. The rule base can be trained under different contexts, represented as fuzzy sets. In the present work, contexts are used to represent different behaviors. Several behaviors can in this way be stored in the same rule base and partly share information. The context that best matches present circumstances can be identified using the predictive model and the robot can in this way automatically identify the most suitable behavior for precent circumstances. The performance of PSL as a method for LFD is evaluated with, and without, contextual information. The results indicate that PSL without contexts can learn and reproduce simple behaviors. The system also successfully identifies the most suitable context in almost all test cases. The robot's ability to reproduce more complex behaviors, with partly overlapping and conflicting information, significantly increases with the use of contexts. The results support a further development of PSL as a component of a dynamic hierarchical system performing control and predictions on several levels of abstraction. 

This thesis covers the implementation of visual algorithms for a robot that is to operate at a smelter furnace. The goal is for the robot to replace a human in the opening, closing and flow regulation process as danger can arise when 1300°C slag flows out of the furnace. A thermal lance is used for opening the furnace which means the robot also has to understand if the lance is burning or not. A heat camera with temperature intervals 0-660°C and 300-2000°C was used to record the furnace during these critical moments which has been used to test different vision and tracking algorithms, such as mean shift and continuously adaptive mean shift. The heat images were filtered to extract only the relevant slag flow part, which then were used to track if slag was flowing, and see how large the slag flow was. Opening of the furnace was possible to identify for both temperature intervals. For the closing of the furnace both intervals were also successful, but the lower interval used a different algorithm for this case to be successful. A relative slag flow has been identified which looks promising for further real life studies. The ignition of the lance result is inconclusive as the data recorded was not fit for analysing this case, though a few conclusions could be made indicating a thermal camera may be unfit to track the thermal lance state.

This paper presents static object detection and segmentation method in videos. In this context, background subtraction BS technique based on the frame difference concept is applied to the identification of static objects. First, we estimate a frame differencing foreground mask by computing the difference of each frame with respect to a static reference frame image. The Mixture of Gaussian MOG method is applied to detect the moving particles and then outcome foreground mask is subtracted from frame differencing mask. Pre-processing techniques are applied to reduce the noise from the scene. Finally, morphological operation and largest connected component analysis are applied to segment the object. The proposed method was effectively validated with two public data sets. The results demonstrate the proposed approach can robustly detect, and segment the static objects without any prior information of tracking.

This paper presents a multistage 3D shape reconstruction system of multiple object scenes by considering the silhouette inconsistencies in the shape-from silhouette SFS method. These inconsistencies are common in multiple view images due to object occlusions in different views, segmentation, and shadows or reflection due to objects or light directions. These factors raise huge challenges when attempting to construct the 3D shape by using existing approaches that reconstruct only that part of the volume which projects consistently in all the silhouettes, leaving the rest unreconstructed. As a result, the final shape is not robust due to multiview objects' occlusion and shadows. In this regard, we consider the primary factors affecting reconstruction by analyzing the multiple images and perform pre-processing steps to optimize the silhouettes. Finally, the 3D shape is reconstructed by using the volumetric approach SFS. Theory and experimental results show that the performance of the modified algorithm was efficiently improved, which can improve the accuracy of the reconstructed shape and be robust to errors in the silhouettes, volume, and computationally inexpensive.

The amount of data measured during a typical manufacturing process is immense. To efficiently utilize these data without becoming overwhelmed with confusing and often conflicting information is difficult to impossible when using traditional univariate methods. Multivariate data mining methods can be used to examine large data sets by extracting relationships between variables to highlight variable correlations and deviations. Specifically, PLS-trees can be used to quickly identify significant clusters in large datasets and to highlight the differences within the groups.

Ball dribbling is a central element of basketball. One main challenge for realizing basketball robots is to stabilize periodic motion of the ball. The task is nontrivial due to the discrete-continuous nature of the corresponding dynamics. This paper proposes to add an elastic element to the manipulator so the ball can be controlled in a continuous-time phase instead of an intermittent contact. Optimal catching and pushing trajectories are planned for the underactuated system based on the virtual holonomic constraints approach. First experimental studies are presented to evaluate the approach. 

In this paper we propose an implementation scheme of independent joint control for a four-degree-of-freedom heavy-duty hydraulic actuated crane. First, on-line generation of feasible trajectories, following a driver's lead and satisfying the actuator constrains for the redundant kinematic chain, is performed. Second, an implementation of two new Sliding Mode algorithms with variable barrier function gains, which allow robust tracking of the generated trajectory with alleviation of high frequency oscillations, is presented. Experimental results are presented to show the effectiveness of the proposed semi-automation scheme, exploiting the forestry application motivated low accuracy requirement.

As robots are introduced to various domains and applications, Human-Robot Teaming (HRT) capabilities are essential. Such capabilities involve teaming with humans in/on/out-the-loop at different levels of abstraction, leveraging the complementing capabilities of humans and robots. This requires robotic systems with the ability to dynamically vary their level or degree of autonomy to collaborate with the human(s) efficiently and overcome various challenging circumstances. Variable Autonomy (VA) is an umbrella term encompassing such research, including but not limited to shared control and shared autonomy, mixed-initiative, adjustable autonomy, and sliding autonomy. This workshop is driven by the timely need to bring together VA-related research and practices that are often disconnected across different communities as the field is relatively young. The workshop's goal is to consolidate research in VA. To this end, and given the complexity and span of Human-Robot systems, this workshop will adopt a holistic trans-disciplinary approach aiming to a) identify and classify related common challenges and opportunities; b) identify the disciplines that need to come together to tackle the challenges; c) identify and define common terminology, approaches, methodologies, benchmarks, and metrics; d) define short- and longterm research goals for the community. To achieve these objectives, this workshop aims to bring together industry stakeholders, researchers from fields under the banner of VA, and specialists from other highly related fields such as human factors and psychology. The workshop will consist of a mix of invited talks, contributed papers, and an interactive discussion panel, toward a shared vision for VA.

Research on robotic ostracism is still scarce and has only explored its effects on adult populations. Although the results revealed important carryover effects of robotic exclusion, there is no evidence yet that those results occur in child-robot interactions. This paper provides the first exploration of robotic ostracism with children. We conducted a study using the Robotic Cyberball Paradigm in a third-person perspective with a sample of 52 children aged between five to ten years old. The experimental study had two conditions: Exclusion and Inclusion. In the Exclusion condition, children observed a peer being excluded by two robots; while in the Inclusion condition, the observed peer interacted equally with the robots. Notably, even 5-year-old children could discern when robots excluded another child. Children who observed exclusion reported lower levels of belonging and control, and exhibited higher prosocial behaviour than those witnessing inclusion. However, no differences were found in children's meaningful existence, self-esteem, and physical proximity across conditions. Our user study provides important methodological considerations for applying the Robotic Cyberball Paradigm with children. The results extend previous literature on both robotic ostracism with adults and interpersonal ostracism with children. We finish discussing the broader implications of children observing ostracism in human-robot interactions.

Over the past few years, there has been a noticeable surge in efforts to design novel tools and approaches that incorporate Artificial Intelligence (AI) into rehabilitation of persons with lower-limb impairments, using robotic exoskeletons. The potential benefits include the ability to implement personalized rehabilitation therapies by leveraging AI for robot control and data analysis, facilitating personalized feedback and guidance. Despite this, there is a current lack of literature review specifically focusing on AI applications in lower-limb rehabilitative robotics. To address this gap, our work aims at performing a review of 37 peer-reviewed papers. This review categorizes selected papers based on robotic application scenarios or AI methodologies. Additionally, it uniquely contributes by providing a detailed summary of input features, AI model performance, enrolled populations, exoskeletal systems used in the validation process, and specific tasks for each paper. The innovative aspect lies in offering a clear understanding of the suitability of different algorithms for specific tasks, intending to guide future developments and support informed decision-making in the realm of lower-limb exoskeleton and AI applications.

Performing work for extended periods of time while using the lowest amount of resources is an important aspect for productivity in many industries. In forestry, the productivity of a forwarder is seen as the volume of material it can extract to a roadside landing in a certain amount of time, where the process of loading and unloading logs represents a large part of the work. During this process, the esnergy consumed by the machine is directly related to the speed of the crane. Thus, increasing productivity implies increasing the operating velocity of cranes. But according to current design of forestry cranes, this conversely leads to an undesired increase in consumption of resources (e.g. fuel). A second method is to alter the machine's design, such as rotating the log bunk. This article considers both methods through a simulation-based comparison aiming to evaluate the energy consumption of two crane-bunk systems when loading. The first simulation system considers a forestry crane with a fixed log bunk (forwarder-like crane). The second simulation system takes into account a forestry crane and a rotating log bunk (harwarder-like crane). The analysis presented considers the fundamental mathematics required to analyze the dynamics of forestry cranes and the principles required to plan energy-optimal motions. The simulation results show that energy savings of 43% to 61% can be obtained by determining energy-optimal motions and using a harwarder-like crane architecture.

Causality is relatively unexplored in robotics even if it is highly relevant, in several respects. In this paper, we study how a robot’s causal understanding can be improved by allowing the robot to ask humans causal questions. We propose a general algorithm for selecting direct causal effects to ask about, given a partial causal representation (using partially directed acyclic graphs, PDAGs) obtained from observational data. We propose three versions of the algorithm inspired by different causal discovery techniques, such as constraint-based, score-based, and interventions. We evaluate the versions in a simulation study and our results show that asking causal questions improves the causal representation over all simulated scenarios. Further, the results show that asking causal questions based on PDAGs discovered from data provides a significant improvement compared to asking questions at random, and the version inspired by score-based techniques performs particularly well over all simulated experiments.

A Ground Penetrating Radar (GPR) is a non-invasive measurement tool to locate objects

in the subsurface. The GPR transmits electromagnetic waves into the ground and

records the waves reflected from surface interfaces of different materials. To accurately

find these surfaces after measuring, it is important to record the precise location of

the GPR and minimize reflected noise. Since a GPR cannot distinguish the direction

from which the waves were reflected, this can result in a misinterpretation of the data

if waves are reflected from surrounding objects. This problem can be reduced by also

mapping objects in the surroundings. The work of this thesis is aimed at implementing

a system that uses a Real-Time Kinematics (RTK) GNSS (Global Navigation Satellite

System) receiver for precise positioning together with a 2D-LiDAR (Light Detection

And Ranging) to record a 3D map of the surroundings. We used the 3D-LiDAR system

to record vertical planes (cross-sections) that were processed into a 3D volume

map. We found that the RTK GNSS receiver performed well and delivered the position

within centimeters when provided with corrections, while it was about 2.5 m off

without corrections. The performance was compared with a professional-grade Leica

RTK receiver and the difference in latitude and longitude ranged from 0.001-0.002 m

and 0.002-0.004 m, respectively. By fusing the RTK position with the LiDAR data using

the software Robot Operating System (ROS), we created 3D maps that represented

the surroundings along the traveled path. Our developed system, consisting of an RTK

GNSS receiver and the 2D LiDAR, gave promising results and we are optimistic that

combining the system with a GPR can improve the interpretation of the subsurface.

Thus, the proposed method seems promising to be used during GPR mapping.

A hierarchical clustering approach based on a set of PLS models is presented. Called PLS-Trees (R), this approach is analogous to classification and regression trees (CART), but uses the scores of PLS regression models as the basis for splitting the clusters, instead of the individual X-variables. The split of one cluster into two is made along the sorted first X-score (t(1)) of a PLS model of the cluster, but may potentially be made along a direction corresponding to a combination of scores. The position of the split is selected according to the improvement of a weighted combination of (a) the variance of the X-score, (b) the variance of Y and (c) a penalty function discouraging an unbalanced split with very different numbers of observations. Cross-validation is used to terminate the branches of the tree, and to determine the number of components of each cluster PLS model. Some obvious extensions of the approach to OPLS-Trees and trees based on hierarchical PLS or OPLS models with the variables divided in blocks depending on their type, are also mentioned. The possibility to greatly reduce the number of variables in each PLS model on the basis of their PLS w-coefficients is also pointed out. The approach is illustrated by means of three examples. The first two examples are quantitative structure-activity relationship (QSAR) data sets, while the third is based on hyperspectral images of liver tissue for identifying different sources of variability in the liver samples.

Forests represent one of the biggest terrestrial ecosystems of Earth, that can produce important raw renewable materials such as wood with the help of sun, air and water. To efficiently extract these raw materials, the tree harvesting process is highly mechanized in developed countries, meaning that advanced forestry machines are continuously used to fell, to process and to transport the logs and biomass obtained from the forests. However, working with these machines is demanding both mentally and physically, which are known factors to negatively affect operator productivity. Mental fatigue is mostly due to the manual operation of the on-board knuckleboom crane, which requires advanced cognitive work with two joystick levers, while the most serious physical strains arise from cabin vibrations. These vibrations are generated from knuckleboom crane vibrations as a result of aggressive manual operation.

To enhance operator workload, well-being, and to increase productivity of the logging process, semi-automation functions are suggested, which are supervised automatic executions of specific work elements. Some of the related issues are addressed in the current thesis. Therefore, the content is divided into: (1) the design and development of a semi-automation function focused only on the base joint actuator (slewing actuator) of a knuckleboom crane, and (2) active vibration damping solutions to treat crane structure vibrations induced by the main lift cylinder (inner boom actuator). The considered reference machine is a downsized knuckleboom crane of a forwarder machine, which is used to pick up log assortments from a harvesting site.

The proposed semi-automation function presented in the first part could be beneficial for operators to use during log loading/unloading scenarios. It consists from a closed-loop position control architecture, to which smooth reference slewing trajectories are provided by a trajectory planner that is automated via operator commands. The used trajectory generation algorithms are taken from conventional robotics and adapted to semi-automation context with proposed modifications that can be customizable by operators.

Further, the proposed active vibration damping solutions are aimed to reduce vibrations of the knuckleboom crane excited by the inner boom actuator due to aggressive manual commands. First, a popular input shaping control technique combined with a practical switching logic was investigated to deal with the excited payload oscillations. This technique proved to be useful with a fixed crane pose, however it did not provide much robustness in terms of different link configurations. To tackle this problem an H2-optimal controller is developed, which is active in the pressure feedback-loop and its solely purpose is to damp the same payload oscillations. During the design process, operator commands are treated and explained from input disturbance viewpoint.

All of the hypothesis throughout this thesis were verified with extensive experimental studies using the reference machine.

Nowadays, forest harvesting is highly mechanized. The commercially available forestry machines are equipped with knuckleboom cranes that are hydraulically actuated, and manually controlled through a set of joysticks and buttons. A common problem that human operators face during manipulation of such knuckleboom cranes, are the crane structure oscillations created by non-smooth or too aggressive manual joystick-based commands. These oscillations not only contribute to actuator wear, but are also dangerous for operators and the environment as well. The current paper investigates the oscillation attenuation induced by the motion of the inner boom actuator and is based on H2-optimal controller synthesis active in the pressure feedback loop. Furthermore, the controller robustness is verified experimentally considering different working conditions of the reference machine, which also verifies the effectiveness of the approach. 

Simplifying the operation of forestry machines with operator-centered semi-automation is needed in the modern timber harvesting industry in order to increase operator productivity and comfort, to reduce learning time of novice operators and to ensure safer manipulation of the cranes. In this paper, useful tools towards operator-centered semi-automation of the base joint actuator of a forwarder crane are proposed. The main goal is to allow comfortable automated motions that do not excite dangerous oscillations of the freely-hanging grapple. Moreover, operator commands are used interactively with a closed-loop position control scheme to assure automated slewing motions. Smooth reference trajectories are provided for the position controller with an on-line trajectory generation algorithm that is developed by combining properties of two standard trajectory generation methods. A practical algorithm based on experiments is introduced to find the trajectory that guaranties minimal grapple oscillations within a set of relatively fast trajectories. Further on, the log loading/unloading tasks are discussed and verified experimentally using the proposed approach on a forwarder crane prototype.

The modern timber harvesting industry would be ineffective without heavy duty advanced machinery used for logging. However, with benefits of mechanization comes the operation complexity. Introducing automation is expected to reduce the mental and physical load on the operator and improve the machine use efficiency. Nonetheless, with current technology fully autonomous timber harvesting is impossible. In this paper a semi-automation scenario is presented using the base joint actuator of a forestry forwarder crane taking into consideration the need to attenuate unwanted oscillations of its hanging grapple. We address the necessary motion planning and motion stabilization tasks. To reduce oscillations along a nominal trajectory, we design smooth reference profiles based on experiments. Meanwhile, a practical structure for a feedback controller is proposed and tested. In this process, actuator nonlinearities are dealt with feasible identification and compensation techniques.

Working with forestry cranes is not easy due to their complex mechanical structure, non-linear behavior of the hydraulic actuation system, and non-intuitive joint-based control; however, with automation, the level of manipulation difficulty can be reduced. This is potentially useful for the operators since they are prone to be more productive if semi-automation functions are introduced to a certain level. In this paper, a semi-automation function for the base joint actuator of a forestry forwarder crane is proposed. The semi-automation function is based on a design of an interactive on-line trajectory generation algorithm with variable final time that acts as a reference signal to a closed-loop position controller. Moreover, the advantage of this scheme is that the operators are kept in the loop by directly being in charge of controlling the final time for the on-line trajectory generation algorithm. Experiments with a downsized industry-standard forwarder crane verify the applicability and advantage of the proposed scheme.

Smooth operation of heavy-duty forestry cranes is not an easy task for the operators with the current joystick-based control method that is complex and non-intuitive. Moreover, abrupt movements of the same joysticks provoke aggressive signals that can lead to oscillatory motions in the actuators and in the entire crane. These oscillations, not only contribute to wear of the joint actuators but also can cause damage to both the operators and the environment; therefore, they must be attenuated. The proposed approach in this paper uses the popular input shaping control technique combined with a practical switching logic to deal with different frequency payload oscillations induced by the motion of the inner boom actuator of a forwarder crane. The results show a significant improvement in terms of visible oscillation reduction monitored through their appearance in the torque signal computed from pressure measurements. Experiments performed on a down-sized forestry crane verifies the effectiveness of the approach.

Building general purpose autonomous robots that suit a wide range of user-specified applications, requires a leap from today's task-specific machines to more flexible and general ones. To achieve this goal, one should move from traditional preprogrammed robots to learning robots that easily can acquire new skills. Learning from Demonstration (LfD) and Imitation Learning (IL), in which the robot learns by observing a human or robot tutor, are among the most popular learning techniques. Showing the robot how to perform a task is often more natural and intuitive than figuring out how to modify a complex control program. However, teaching robots new skills such that they can reproduce the acquired skills under any circumstances, on the right time and in an appropriate way, require good understanding of all challenges in the field. Studies of imitation learning in humans and animals show that several cognitive abilities are engaged to learn new skills correctly. The most remarkable ones are the ability to direct attention to important aspects of demonstrations, and adapting observed actions to the agents own body. Moreover, a clear understanding of the demonstrator's intentions and an ability to generalize to new situations are essential. Once learning is accomplished, various stimuli may trigger the cognitive system to execute new skills that have become part of the robot's repertoire. The goal of this thesis is to develop methods for learning from demonstration that mainly focus on understanding the tutor's intentions, and recognizing which elements of a demonstration need the robot's attention. An architecture containing required cognitive functions for learning and reproduction of high-level aspects of demonstrations is proposed. Several learning methods for directing the robot's attention and identifying relevant information are introduced. The architecture integrates motor actions with concepts, objects and environmental states to ensure correct reproduction of skills. Another major contribution of this thesis is methods to resolve ambiguities in demonstrations where the tutor's intentions are not clearly expressed and several demonstrations are required to infer intentions correctly. The provided solution is inspired by human memory models and priming mechanisms that give the robot clues that increase the probability of inferring intentions correctly. In addition to robot learning, the developed techniques are applied to a shared control system based on visual servoing guided behaviors and priming mechanisms. The architecture and learning methods are applied and evaluated in several real world scenarios that require clear understanding of intentions in the demonstrations. Finally, the developed learning methods are compared, and conditions where each of them has better applicability are discussed.

Learning techniques are drawing extensive attention in the robotics community. Some reasons behind moving from traditional preprogrammed robots to more advanced human fashioned techniques are to save time and energy, and allow non-technical users to easily work with robots. Learning from Demonstration (LfD) and Imitation Learning (IL) are among the most popular learning techniques to teach robots new skills by observing a human or robot tutor.

Flawlessly teaching robots new skills by LfD requires good understanding of all challenges in the field. Studies of imitation learning in humans and animals show that several cognitive abilities are engaged to correctly learn new skills. The most remarkable ones are the ability to direct attention to important aspects of demonstrations, and adapting observed actions to the agents own body. Moreover, a clear understanding of the demonstrator's intentions is essential for correctly and completely replicating the behavior with the same effects on the world. Once learning is accomplished, various stimuli may trigger the cognitive system to execute new skills that have become part of the repertoire.

Considering identified main challenges, the current thesis attempts to model imitation learning in robots, mainly focusing on understanding the tutor's intentions and recognizing what elements of the demonstration need the robot's attention. Thereby, an architecture containing required cognitive functions for learning and reproducing high-level aspects of demonstrations is proposed. Several learning methods for directing the robot's attention and identifying relevant information are introduced. The architecture integrates motor actions with concepts, objects and environmental states to ensure correct reproduction of skills. This is further applied in learning object affordances, behavior arbitration and goal emulation.

The architecture and learning methods are applied and evaluated in several real world scenarios that require clear understanding of goals and what to look for in the demonstrations. Finally, the developed learning methods are compared, and conditions where each of them has better applicability is specified.

Most of the humans day to day tasks include sequences ofactions that lead to a desired goal. In domains which humans are replacedby robots, the ability of learning new skills easy and fast plays animportant role. The aim of this research paper is to incorporate sequentiallearning into Learning from Demonstration (LfD) in an architecturewhich mainly focuses on high-level representation of behaviors. The primarygoal of the research is to investigate the possibility of utilizingSemantic Networks in order to enable the robot to learn new skills insequences.

In this paper we present an approach for high-level behavior recognition and selection integrated with alow-level controller to help the robot to learn new skills from demonstrations. By means of SemanticNetwork as the core of the method, the robot gains the ability to model the world with concepts and relatethem to low-level sensory-motor states. We also show how the generalization ability of Semantic Networkscan be used to extend learned skills to new situations.

This paper gives a brief overview of challenges indesigning cognitive architectures for Learning fromDemonstration. By investigating features and functionality ofsome related architectures, we propose a modular architectureparticularly suited for sequential learning high-levelrepresentations of behaviors. We head towards designing andimplementing goal based imitation learning that not only allowsthe robot to learn necessary conditions for executing particularbehaviors, but also to understand the intents of the tutor andreproduce the same behaviors accordingly.

In many robotics shared control applications, users are forced to focus hard on the robot due to the task’s high sensitivity or the robot’s misunderstanding of the user’s intention. This brings frustration and dissatisfaction to the user and reduces overall efficiency. The user’s intention is sometimes unclear and hard to identify without some kind of bias in the identification process. In this paper, we present a solution in which an attentional mechanism helps the robot to recognize the user’s intention. The solution uses a priming mechanism and parameterized behavior primitives to support intention recognition and improve shared control for teleoperation tasks.

Abstract Robotics is tightly connected to both artificial intelligence (AI) and control theory. Both AI and control based robotics are active and successful research areas, but research is often conducted by well separated communities. In this paper, we compare the two approaches in a case study for the design of a robot that should move its arm towards an object with the help of camera data. The control based approach is a model-free version of Image Based Visual Servoing (IBVS), which is based on mathematical modeling of the sensing and motion task. The AI approach, here denoted Behavior-Based Visual Servoing (BBVS), contains elements that are biologically plausible and inspired by schema-theory. We show how the two approaches lead to very similar solutions, even identical given a few simplifying assumptions. This similarity is shown both analytically and numerically. However, in a simple picking task with a 3 DoF robot arm, BBVS shows significantly higher performance than the IBVS approach, partly because it contains more manually tuned parameters. While the results obviously do not apply to all tasks and solutions, it illustrates both strengths and weaknesses with both approaches, and how they are tightly connected and share many similarities despite very different starting points and methodologies.

Learning from Demonstration (LfD) is an established robot learning technique by which a robot acquires a skill by observing a human or robot teacher demonstrating the skill. In this paper we address the ambiguity involved in inferring the intention with one or several demonstrations. We suggest a method based on priming, and a memory model with similarities to human learning. Conducted experiments show that the developed method leads to faster and improved understanding of the intention with a demonstration by reducing ambiguity.