umu.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Human brains and virtual realities: Computer-generated presence in theory and practice
Umeå University, Faculty of Science and Technology, Department of Computing Science.
2013 (English)Doctoral thesis, comprehensive summary (Other academic)Alternative title
Mänskliga hjärnor och virtuella verkligheter : Datorgenererad närvaro i teori och praktik (Swedish)
Abstract [en]

A combined view of the human brain and computer-generated virtual realities is motivated by recent developments in cognitive neuroscience and human-computer interaction (HCI). The emergence of new theories of human brain function, together with an increasing use of realistic human-computer interaction, give reason to believe that a better understanding of the relationship between human brains and virtual realities is both possible and valuable. The concept of “presence”, described as the subjective feeling of being in a place that feels real, can serve as a cornerstone concept in the development of such an understanding, as computer-generated presence is tightly related to how human brains work in virtual realities.

In this thesis, presence is related both to theoretical discussions rooted in theories of human brain function, and to measurements of brain activity during realistic interaction. The practical implications of such results are further developed by considering potential applications. This includes the development and evaluation of a prototype application, motivated by presented principles.

The theoretical conception of presence in this thesis relies on general principles of brain function, and describes presence as a general cognitive function, not specifically related to virtual realities. Virtual reality (VR) is an excellent technology for investigating and taking advantage of all aspects of presence, but a more general interpretation allows the same principles to be applied to a wide range of applications.

Functional magnetic resonance imaging (fMRI) was used to study the working human brain in VR. Such data can inform and constrain further discussion about presence. Using two different experimental designs we have investigated both the effect of basic aspects of VR interaction, as well as the neural correlates of disrupted presence in a naturalistic environment.

Reality-based brain-computer interaction (RBBCI) is suggested as a concept for summarizing the motivations for, and the context of, applications building on an understanding of human brains in virtual realities. The RBBCI prototype application we developed did not achieve the set goals, but much remains to be investigated and lessons from our evaluation point to possible ways forward. A developed use of methods and techniques from computer gaming is of particular interest.

Abstract [sv]

Ett kombinerat perspektiv på den mänskliga hjärnan och datorgenererade virtuella verkligheter motiveras av den senaste utvecklingen inom kognitiv neurovetenskap och människa-datorinteraktion (MDI). Framväxten av nya teorier om den mänskliga hjärnan, tillsammans med en ökande användning av realistisk människa-datorinteraktion, gör det troligt att en bättre förståelse för relationen mellan mänskliga hjärnor och virtuella verkligheter är både möjlig och värdefull. Begreppet "närvaro", som i detta sammanhang beskrivs som den subjektiva känslan av att vara på en plats som känns verklig, kan fungera som en hörnsten i utvecklingen av en sådan förståelse, då datorgenererad närvaro är tätt kopplat till hur mänskliga hjärnor fungerar i virtuella verkligheter.

I denna avhandling kopplas närvaro både till teoretiska diskussioner grundade i teorier om den mänskliga hjärnan, och till mätningar av hjärnans aktivitet under realistisk interaktion. De praktiska konsekvenserna av sådana resultat utvecklas vidare med en närmare titt på potentiella tillämpningar. Detta inkluderar utveckling och utvärdering av en prototypapplikation, motiverad av de presenterade principerna.

Den teoretiska diskussionen av närvaro i denna avhandling bygger på allmänna principer för hjärnans funktion, och beskriver känslan av närvaro som en generell kognitiv funktion, inte specifikt relaterad till virtuella verkligheter. Virtuell verklighet (virtual reality, VR) är en utmärkt teknik för att undersöka och dra nytta av alla aspekter av närvaro, men en mer allmän tolkning gör att samma principer kan tillämpas på ett brett spektrum av applikationer.

Funktionell hjärnavbildning (fMRI) användes för att studera den arbetande mänskliga hjärnan i VR. Sådant data kan informera och begränsa en vidare diskussion av närvaro. Med hjälp av två olika försöksdesigner har vi har undersökt både effekten av grundläggande aspekter av VR-interaktion, och neurala korrelat av störd närvaro i en naturalistisk miljö.

Verklighets-baserad hjärna-dator interaktion (reality-based brain-computer interaction, RBBCI) föreslås som ett begrepp för att sammanfatta motiv och kontext för applikationer som bygger på en förståelse av den mänskliga hjärnan i virtuella verkligheter. Den prototypapplikation vi utvecklade uppnådde inte de uppsatta målen, men mycket återstår att utforska och lärdomar från vår utvärdering pekar på möjliga vägar framåt. En vidare användning av metoder och tekniker från dataspel är speciellt intressant.

Place, publisher, year, edition, pages
Umeå: Umeå universitet , 2013. , 43 p.
Series
Report / UMINF, ISSN 0348-0542 ; 13.06
Keyword [en]
Human brain function, virtual reality, presence, cognitive neuroscience, human-computer interaction, brain imaging, fMRI, BCI, neural correlates, HCI theory, reality-based interaction.
National Category
Human Computer Interaction
Identifiers
URN: urn:nbn:se:umu:diva-68664ISBN: 978-91-7459-617-5 (print)ISBN: 978-91-7459-618-2 (print)OAI: oai:DiVA.org:umu-68664DiVA: diva2:617309
Public defence
2013-05-17, Naturvetarhuset, N360, Umeå universitet, Umeå, 13:15 (English)
Opponent
Supervisors
Available from: 2013-04-23 Created: 2013-04-22 Last updated: 2013-04-23Bibliographically approved
List of papers
1. Mind the brain: The Potential of Basic Principles for Brain Function and Interaction
Open this publication in new window or tab >>Mind the brain: The Potential of Basic Principles for Brain Function and Interaction
2013 (English)Report (Other academic)
Abstract [en]

The prevalence and complexity of human-computer interaction makes a general understanding of human cognition important in design and development. Knowledge of some basic, relatively simple, principles for human brain function can significantly help such understanding in the interdisciplinary field of research and development Human-Computer Interaction (HCI) where no one can be an expert at everything. This paper explains a few such principles, relates them to human-computer interaction, and illustrates their potential. Most of these ideas are not new, but wider appreciation of the potential power of basic principles is only recently emerging as a result of developments within cognitive neuroscience and information theory. The starting point in this paper is the concept of mental simulation. Important and useful properties of mental simulations are explained using basic principles such as the free-energy principle. These concepts and their properties are further related to HCI by drawing on similarities to the theoretical framework of activity theory. Activity theory is particularly helpful to relate simple but abstract principles to real world applications and larger contexts. Established use of activity theory as a theoretical framework for HCI also exemplifies how theory may benefit HCI in general. Briefly, two basic principles that permeate this perspective are: the need for new skills and knowledge to build upon and fit into what is already there (grounding) and the importance of predictions and prediction errors (simulation).

Series
Report / UMINF, ISSN 0348-0542 ; 13.04
Keyword
HCI theory concepts and models, brain function, activity theory, grounded cognition, mental simulation, the free-energy principle
National Category
Human Computer Interaction
Identifiers
urn:nbn:se:umu:diva-68659 (URN)
Available from: 2013-04-22 Created: 2013-04-22 Last updated: 2013-06-03Bibliographically approved
2. Presence and general principles of brain function
Open this publication in new window or tab >>Presence and general principles of brain function
2012 (English)In: Interacting with computers, ISSN 0953-5438, E-ISSN 1873-7951, Vol. 24, no 4 SI, 193-202 p.Article in journal (Refereed) Published
Abstract [en]

Recent developments in general theories of cognition and brain function make it possible to consider the concept of presence from a new perspective, based in general principles of brain function. The importance of interaction with reality for the development and function of the brain and human cognition is increasingly emphasized. The brain is explained as implementing a generative model of the current environment. Whether this environment is real or virtual does not matter. Mental simulations are created for whatever one interacts with, when possible. This view provides a basis for relating human experiences in virtual environments to several theories that explain cognition and brain function on many levels, from ultimate evolutionary motivations to plausible neural implementations. The purpose of this paper is not to provide yet another definition of presence but to suggest explanations of phenomena commonly related to presence, with a basis in general principles of brain function. Such principles are employed to explain how, and why, interaction with our environment, and internalization of objects and tools therein, play an essential role in human cognition. This provides a rich basis for further analysis of how central aspects of presence, such as breaks in presence or the perceptual illusion of non-mediation, may work on a fundamental level. More general descriptions of such phenomena have advantages such as being easier to relate to new contexts and technologies, and opening up for additional inspiration and confirmation from other disciplines such as cognitive neuroscience. In addition to an account of general principles for brain function and a discussion about the concept of presence in light of these, this paper also relates this discussion to a number of previous accounts of presence, and to practical implications and applications for interaction design.

Keyword
Presence, Brain function, Virtual reality, Cognitive neuroscience
National Category
Human Computer Interaction
Identifiers
urn:nbn:se:umu:diva-56580 (URN)10.1016/j.intcom.2012.04.004 (DOI)000309847600004 ()
Available from: 2012-06-20 Created: 2012-06-20 Last updated: 2017-12-07Bibliographically approved
3. Effects of interactivity and 3D-motion on mental rotation brain activity in an immersive virtual environment
Open this publication in new window or tab >>Effects of interactivity and 3D-motion on mental rotation brain activity in an immersive virtual environment
Show others...
2010 (English)In: Proceedings of the 28th international conference on Human factors in computing systems, Association for Computing Machinery (ACM), 2010, 869-878 p.Conference paper, Published paper (Refereed)
Abstract [en]

The combination of virtual reality (VR) and brain measurements is a promising development of HCI, but the maturation of this paradigm requires more knowledge about how brain activity is influenced by parameters of VR applications. To this end we investigate the influence of two prominent VR parameters, 3d-motion and interactivity, while brain activity is measured for a mental rotation task, using functional MRI (fMRI). A mental rotation network of brain areas is identified, matching previous results. The addition of interactivity increases the activation in core areas of this network, with more profound effects in frontal and preparatory motor areas. The increases from 3d-motion are restricted to primarily visual areas. We relate these effects to emerging theories of cognition and potential applications for brain-computer interfaces (BCIs). Our results demonstrate one way to provoke increased activity in task-relevant areas, making it easier to detect and use for adaptation and development of HCI.

Place, publisher, year, edition, pages
Association for Computing Machinery (ACM), 2010
Keyword
Virtual reality, fMRI, Reality-based interaction, VRfMRI, brain imaging, BCI
National Category
Computer Science
Identifiers
urn:nbn:se:umu:diva-40034 (URN)10.1145/1753326.1753454 (DOI)000281276700097 ()978-1-60558-929-9 (ISBN)
Conference
CHI 2010 - ACM Conference on Human Factors in Computing Systems, 10-15 Apri, Atlanta, GA, USA
Available from: 2011-02-15 Created: 2011-02-15 Last updated: 2015-10-09Bibliographically approved
4. Capturing neural correlates of disrupted presence ina naturalistic virtual environment
Open this publication in new window or tab >>Capturing neural correlates of disrupted presence ina naturalistic virtual environment
2013 (English)Report (Other academic)
Abstract [en]

The concept of presence is commonly related to whether or not a user feels, acts, and reacts as if he/she were in a real familiar environment when using a virtual reality (VR) application. Understanding the neural correlates of presence may provide a foundation for objective measurements of presence and important constraints for theoretical explanations of presence. Discussions about the neural basis for presence are relatively common, but brain imaging has rarely been applied to investigating this issue. Previous studies have focused on detecting average differences between conditions that correlate with differences in reported presence. In this study we focused on breaks in presence and associated periods of disrupted presence as an important complement to previous work. Specifically, we measured brain activity with functional magnetic resonance imaging (fMRI) during execution of an everyday task in a naturalistic virtual environment (VE). Time periods of disrupted presence were identified by subject reports indicating something strange in the current environment, interpreted as a violation of expectations related to the sense of presence. Disrupted presence was associated with increased activity in the frontopolar cortex (FPC), lateral occipito-temporal cortex (LOTC), the temporal poles (TP), and the posterior superior temporal cortex (pSTC). We relate these areas to integration of key aspects of a presence experience, relating the (changing) situation to management of task and goals (FPC), interpretation of visual input (LOTC), emotional evaluation of the context (TP) and possible interactions (pSTC). Modulation of the activity level in these brain areas is consistent with an interpretation of disrupted presence as a re-evaluation of key aspects of a subjective mental reality, updating the synchronization with the virtual environment as previous predictions fail. Such a subjective mental reality may also be related to a self-centered type of mentalization, providing a link to accounts of presence building on the self.

Series
Report / UMINF, ISSN 0348-0542 ; 13.05
Keyword
Presence, brain function, virtual reality, functional MRI
National Category
Human Computer Interaction Neurosciences
Identifiers
urn:nbn:se:umu:diva-68662 (URN)
Available from: 2013-04-22 Created: 2013-04-22 Last updated: 2013-06-03Bibliographically approved
5. Adaptive games for cognitive training: Lessons measuring arousal with EEG
Open this publication in new window or tab >>Adaptive games for cognitive training: Lessons measuring arousal with EEG
2013 (English)Conference paper, Published paper (Refereed)
Abstract [en]

Computerized cognitive training is an area where the use of computer games technology and methods has a great potential, for example, to address cognitive decline in an aging population. Adaptive games, in particular, are of great interest as the level of training has often been suggested as important for efficient training. An important part of any adaptive application is measuring and interpreting whatever the game should adapt to. In this paper we describe our work on using the Emotiv Epoc commercial EEG headset in order to measure and adapt to the user's level of arousal in two different applications. The first application is an adaptation of a classic cognitive training task (N-back) using game technologies to create a dynamic and (relatively) realistic version in a 3d-environment. The second application is a simple version of the classic space invaders game. In both applications EEG measurements recorded during initial training are used in a later phase to adapt the difficulty of the game automatically. While we managed to get this setup to work to a limited degree for some individuals, we failed to create a system where this method worked reliably across subjects and trials. In this paper, we describe what we tried, what worked, and some of the lessons we learned.

Keyword
Adaptive games, brain-computer interfaces, cognitive training
National Category
Human Computer Interaction
Identifiers
urn:nbn:se:umu:diva-68663 (URN)
Conference
CHI 2013 Workshop on Games User Research
Available from: 2013-04-22 Created: 2013-04-22 Last updated: 2013-04-23Bibliographically approved

Open Access in DiVA

Human brains and virtual realities(5137 kB)962 downloads
File information
File name FULLTEXT02.pdfFile size 5137 kBChecksum SHA-512
49a042d6b10fff918e636f56ab275ab805013b9f27b756ba3411442e9802785a33cf74ca6ea1b60f6a58b759919581420b228893dd687cef3688c8a0ef113641
Type fulltextMimetype application/pdf

Search in DiVA

By author/editor
Sjölie, Daniel
By organisation
Department of Computing Science
Human Computer Interaction

Search outside of DiVA

GoogleGoogle Scholar
Total: 962 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 1282 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf