umu.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
A formalism for learning from demonstration
Umeå University, Faculty of Science and Technology, Department of Computing Science.
Umeå University, Faculty of Science and Technology, Department of Computing Science.
2010 (English)In: Paladyn Journal of Behavioral Robotics, ISSN 2080-9778, 2081-4836 (e-version), Vol. 1, no 1, 1-13 p.Article in journal (Refereed) Published
Abstract [en]

The paper describes and formalizes the concepts and assumptions involved in Learning from Demonstration (LFD), a common learning technique used in robotics. LFD-related concepts like goal, generalization, and repetition are here defined, analyzed, and put into context. Robot behaviors are described in terms of trajectories through information spaces and learning is formulated as mappings between some of these spaces. Finally, behavior primitives are introduced as one example of good bias in learning, dividing the learning process into the three stages of behavior segmentation, behavior recognition, and behavior coordination. The formalism is exemplified through a sequence learning task where a robot equipped with a gripper arm is to move objects to specific areas. The introduced concepts are illustrated with special focus on how bias of various kinds can be used to enable learning from a single demonstration, and how ambiguities in demonstrations can be identified and handled.

Place, publisher, year, edition, pages
Versita , 2010. Vol. 1, no 1, 1-13 p.
Keyword [en]
Learning from demonstration, ambiguities, behavior, bias, generalization, robot learning
National Category
Human Computer Interaction Computer Science
Research subject
Computer Science
Identifiers
URN: urn:nbn:se:umu:diva-32492DOI: 10.2478/s13230-010-0001-5OAI: oai:DiVA.org:umu-32492DiVA: diva2:303587
Note
co-published with Springer-Verlag GmbH; Published online: 31 March 2010Available from: 2010-06-24 Created: 2010-03-14 Last updated: 2012-01-04Bibliographically approved
In thesis
1. Cognition reversed: Robot learning from demonstration
Open this publication in new window or tab >>Cognition reversed: Robot learning from demonstration
2009 (English)Licentiate thesis, comprehensive summary (Other academic)
Abstract [en]

The work presented in this thesis investigates techniques for learning from demonstration (LFD). LFD is a well established approach to robot learning, where a teacher demonstrates a behavior to a robot pupil. This thesis focuses on LFD where a human teacher demonstrates a behavior by controlling the robot via teleoperation. The robot should after demonstration be able to execute the demonstrated behavior under varying conditions.

Several views on representation, recognition and learning of robot behavior are presented and discussed from a cognitive and computational perspective. LFD-related concepts such as behavior, goal, demonstration, and repetition are defined and analyzed, with focus on how bias is introduced by the use of behavior primitives. This analysis results in a formalism where LFD is described as transitions between information spaces. Assuming that the behavior recognition problem is partly solved, ways to deal with remaining ambiguities in the interpretation of a demonstration are proposed.

A total of five algorithms for behavior recognition are proposed and evaluated, including the dynamic temporal difference algorithm Predictive Sequence Learning (PSL). PSL is model-free in the sense that it makes few assumptions of what is to be learned. One strength of PSL is that it can be used for both robot control and recognition of behavior. While many methods for behavior recognition are concerned with identifying invariants within a set of demonstrations, PSL takes a different approach by using purely predictive measures. This may be one way to reduce the need for bias in learning. PSL is, in its current form, subjected to combinatorial explosion as the input space grows, which makes it necessary to introduce some higher level coordination for learning of complex behaviors in real-world robots.

The thesis also gives a broad introduction to computational models of the human brain, where a tight coupling between perception and action plays a central role. With the focus on generation of bias, typical features of existing attempts to explain humans' and other animals' ability to learn are presented and analyzed, from both a neurological and an information theoretic perspective. Based on this analysis, four requirements for implementing general learning ability in robots are proposed. These requirements provide guidance to how a coordinating structure around PSL and similar algorithms should be implemented in a model-free way.

Abstract [sv]

Arbetet som presenteras i den här avhandlingen undersöker tekniker för att lära robotar från demonstrationer (LFD). LFD är en väl etablerad teknik där en lärare visar roboten hur den ska göra. Den här avhandlingen fokuserar på LFD där en människa fjärrstyr roboten, som i sin tur tolkar demonstrationen så att den kan repetera beteendet vid ett senare tillfälle, även då omgivningen förändrats. Flera perspektiv på representation, igenkänning och inlärning av beteende presenteras och diskuteras från ett kognitionsvetenskaplig och datavetenskapligt perspektiv. LFD-relaterade koncept så som beteende, mål, demonstration och repetition definieras och analyseras, med fokus på hur förkunskap kan implementeras genom beteendeprimitiv. Analysen resulterar i en formalism där LFD beskrivs som övergångar mellan informationsrymder. I termer av formalismen föreslås även sätt att hantera tvetydigheter efter att en demonstration tolkats genom igenkänning av beteendeprimitiv.

Fem algoritmer för igenkänning av beteende presenteras och utvärderas, däribland algoritmen Predictive Sequence Learning (PSL). PSL är modellfri i bemärkelsen att den introducerar få antaganden om inlärningssituationen. PSL kan fungera som en algoritm för både kontroll och igenkänning av beteende. Till skillnad från flertalet tekniker för igenkänning av beteende använder PSL inte likheter i beteende mellan demonstrationer. PSL utnyttjar i stället prediktiva mått som kan minska behovet av domänkunskap vid inlärning. Ett problem med algoritmen är dock att den drabbas av kombinatorisk explosion då inputrymden växer vilket gör att någon form av högre koordination behövs för inlärning av komplexa beteenden.

Avhandlingen ger dessutom en introduktion till beräkningsmässiga modeller av hjärnan där en stark koppling mellan perception och handling spelar en central roll. Typiska egenskaper hos dessa modeller presenters och analyseras från ett neurologiskt och informationsteoretiskt perspektiv. Denna analys resulterar i fyra krav för att implementera generell inlärningsförmåga i robotar. Dessa krav ger vägledning till hur en koordinerande struktur för PSL och liknande algoritmer skulle kunna implementeras på ett modellfritt sätt.

Place, publisher, year, edition, pages
Umeå: Umeå universitet, Institutionen för datavetenskap, 2009. 138 p.
Series
Report / UMINF, ISSN 0348-0542 ; 09:20
National Category
Computer Science Human Computer Interaction
Research subject
Computer Science
Identifiers
urn:nbn:se:umu:diva-32494 (URN)978-91-7264-925-5 (ISBN)
Presentation
2009-12-16, Naturvetarhuset, N300, Umeå Universitet, Umeå, 13:00 (Swedish)
Opponent
Supervisors
Available from: 2010-03-15 Created: 2010-03-14 Last updated: 2010-03-15Bibliographically approved
2. Cognition Rehearsed: Recognition and Reproduction of Demonstrated Behavior
Open this publication in new window or tab >>Cognition Rehearsed: Recognition and Reproduction of Demonstrated Behavior
2012 (English)Doctoral thesis, comprehensive summary (Other academic)
Alternative title[sv]
Robotövningar : Igenkänning och återgivande av demonstrerat beteende
Abstract [en]

The work presented in this dissertation investigates techniques for robot Learning from Demonstration (LFD). LFD is a well established approach where the robot is to learn from a set of demonstrations. The dissertation focuses on LFD where a human teacher demonstrates a behavior by controlling the robot via teleoperation. After demonstration, the robot should be able to reproduce the demonstrated behavior under varying conditions. In particular, the dissertation investigates techniques where previous behavioral knowledge is used as bias for generalization of demonstrations.

The primary contribution of this work is the development and evaluation of a semi-reactive approach to LFD called Predictive Sequence Learning (PSL). PSL has many interesting properties applied as a learning algorithm for robots. Few assumptions are introduced and little task-specific configuration is needed. PSL can be seen as a variable-order Markov model that progressively builds up the ability to predict or simulate future sensory-motor events, given a history of past events. The knowledge base generated during learning can be used to control the robot, such that the demonstrated behavior is reproduced. The same knowledge base can also be used to recognize an on-going behavior by comparing predicted sensor states with actual observations. Behavior recognition is an important part of LFD, both as a way to communicate with the human user and as a technique that allows the robot to use previous knowledge as parts of new, more complex, controllers.

In addition to the work on PSL, this dissertation provides a broad discussion on representation, recognition, and learning of robot behavior. LFD-related concepts such as demonstration, repetition, goal, and behavior are defined and analyzed, with focus on how bias is introduced by the use of behavior primitives. This analysis results in a formalism where LFD is described as transitions between information spaces. Assuming that the behavior recognition problem is partly solved, ways to deal with remaining ambiguities in the interpretation of a demonstration are proposed.

The evaluation of PSL shows that the algorithm can efficiently learn and reproduce simple behaviors. The algorithm is able to generalize to previously unseen situations while maintaining the reactive properties of the system. As the complexity of the demonstrated behavior increases, knowledge of one part of the behavior sometimes interferes with knowledge of another parts. As a result, different situations with similar sensory-motor interactions are sometimes confused and the robot fails to reproduce the behavior.

One way to handle these issues is to introduce a context layer that can support PSL by providing bias for predictions. Parts of the knowledge base that appear to fit the present context are highlighted, while other parts are inhibited. Which context should be active is continually re-evaluated using behavior recognition. This technique takes inspiration from several neurocomputational models that describe parts of the human brain as a hierarchical prediction system. With behavior recognition active, continually selecting the most suitable context for the present situation, the problem of knowledge interference is significantly reduced and the robot can successfully reproduce also more complex behaviors.

Place, publisher, year, edition, pages
Umeå: Department of Computing Science, Umeå University, 2012. 30 p.
Series
Report / UMINF, ISSN 0348-0542 ; 11.16
Keyword
Behavior Recognition, Learning and Adaptive Systems, Learning from Demonstration, Neurocomputational Modeling, Robot Learning
National Category
Computer Vision and Robotics (Autonomous Systems)
Research subject
Computer and Information Science
Identifiers
urn:nbn:se:umu:diva-50980 (URN)978-91-7459-349-5 (ISBN)
Public defence
2012-01-26, S1031, Norra Beteendevetarhuset, Umeå Universitet, Umeå, 13:15 (English)
Opponent
Supervisors
Available from: 2012-01-04 Created: 2012-01-03 Last updated: 2012-01-04Bibliographically approved

Open Access in DiVA

fulltext(636 kB)478 downloads
File information
File name FULLTEXT01.pdfFile size 636 kBChecksum SHA-512
f744e63d2505efc9732dda00388732221dca763b1201e4fc34384ccee964f7662689bbd8be1d82edfe544d052e4889e78a5a8ecfb2e726994d109339e33ed6de
Type fulltextMimetype application/pdf

Other links

Publisher's full text

Authority records BETA

Billing, ErikHellström, Thomas

Search in DiVA

By author/editor
Billing, ErikHellström, Thomas
By organisation
Department of Computing Science
Human Computer InteractionComputer Science

Search outside of DiVA

GoogleGoogle Scholar
Total: 478 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 1369 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf