umu.sePublications
Change search
Link to record
Permanent link

Direct link
BETA
Khan, Muhammad Sikandar LalORCID iD iconorcid.org/0000-0002-3037-4244
Alternative names
Publications (10 of 19) Show all publications
Khan, M. S., Halawani, A., ur Réhman, S. & Li, H. (2018). Action Augmented Real Virtuality Design for Presence. IEEE Transactions on Cognitive and Developmental Systems, 10(4), 961-972
Open this publication in new window or tab >>Action Augmented Real Virtuality Design for Presence
2018 (English)In: IEEE Transactions on Cognitive and Developmental Systems, ISSN 2379-8920, Vol. 10, no 4, p. 961-972Article in journal (Refereed) Published
Abstract [en]

This paper addresses the important question of how to design a video teleconferencing setup to increase the experience of spatial and social presence. Traditional video teleconferencing setups are lacking in presenting the nonverbal behaviors that humans express in face-to-face communication, which results in decrease in presence-experience. In order to address this issue, we first present a conceptual framework of presence for video teleconferencing. We introduce a modern presence concept called real virtuality and propose a new way of achieving this based on body or artifact actions to increase the feeling of presence, and we named this concept presence through actions. Using this new concept, we present the design of a novel action-augmented real virtuality prototype that considers the challenges related to the design of an action prototype, action embodiment, and face representation. Our action prototype is a telepresence mechatronic robot (TEBoT), and action embodiment is through a head-mounted display (HMD). The face representation solves the problem of face occlusion introduced by the HMD. The novel combination of HMD, TEBoT, and face representation algorithm has been tested in a real video teleconferencing scenario for its ability to solve the challenges related to spatial and social presence. We have performed a user study where the invited participants were requested to experience our novel setup and to compare it with a traditional video teleconferencing setup. The results show that the action capabilities not only increase the feeling of spatial presence but also increase the feeling of social presence of a remote person among local collaborators.

Keywords
Real virtuality, Virtual reality, Embodiment, Telepresence, Actions, Perception, Embodied telepresence system, Webrtc, Face occlusion, Face retrieval.
National Category
Signal Processing Computer Systems
Research subject
Computing Science; Computerized Image Analysis
Identifiers
urn:nbn:se:umu:diva-138278 (URN)10.1109/TCDS.2018.2828865 (DOI)000452636400012 ()
Available from: 2017-08-16 Created: 2017-08-16 Last updated: 2019-01-07Bibliographically approved
Khan, M. S., ur Réhman, S., Mi, Y., Naeem, U., Beskow, J. & Li, H. (2017). Moveable facial features in a Social Mediator. In: Beskow J., Peters C., Castellano G., O'Sullivan C., Leite I., Kopp S. (Ed.), Intelligent Virtual Agents: IVA 2017. Paper presented at 17th International Conference on Intelligent Virtual Agents (IVA 2017), Stockholm, August 27-30, 2017. (pp. 205-208). Springer London
Open this publication in new window or tab >>Moveable facial features in a Social Mediator
Show others...
2017 (English)In: Intelligent Virtual Agents: IVA 2017 / [ed] Beskow J., Peters C., Castellano G., O'Sullivan C., Leite I., Kopp S., Springer London, 2017, p. 205-208Conference paper, Poster (with or without abstract) (Refereed)
Abstract [en]

A brief display of facial features based behavior has a majorimpact on personality perception in human-human communications.Creating such personality traits and representations in a social robot isa challenging task. In this paper, we propose an approach for a roboticface presentation based on moveable 2D facial features and present acomparative study when a synthesized face is projected using three setups;1) 3D mask, 2) 2D screen, and 3) our 2D moveable facial featurebased visualization. We found that robot’s personality and character ishighly influenced by the projected face quality as well as the motion offacial features.

Place, publisher, year, edition, pages
Springer London, 2017
Series
Lecture Notes in Computer Science, ISSN 0302-9743, E-ISSN 1611-3349 ; 10498
Keywords
Social robots, telepresence system, facial features, feature tracking, face robot
National Category
Engineering and Technology Computer Systems Signal Processing
Research subject
Computer Science; design
Identifiers
urn:nbn:se:umu:diva-138276 (URN)10.1007/978-3-319-67401-8_23 (DOI)000455400000023 ()978-3-319-67400-1 (ISBN)978-3-319-67401-8 (ISBN)
Conference
17th International Conference on Intelligent Virtual Agents (IVA 2017), Stockholm, August 27-30, 2017.
Available from: 2017-08-16 Created: 2017-08-16 Last updated: 2019-09-05Bibliographically approved
Khan, M. S. (2017). Presence through actions: theories, concepts, and implementations. (Doctoral dissertation). Umeå: Umeå universitet
Open this publication in new window or tab >>Presence through actions: theories, concepts, and implementations
2017 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

During face-to-face meetings, humans use multimodal information, including verbal information, visual information, body language, facial expressions, and other non-verbal gestures. In contrast, during computer-mediated-communication (CMC), humans rely either on mono-modal information such as text-only, voice-only, or video-only or on bi-modal information by using audiovisual modalities such as video teleconferencing. Psychologically, the difference between the two lies in the level of the subjective experience of presence, where people perceive a reduced feeling of presence in the case of CMC. Despite the current advancements in CMC, it is still far from face-to-face communication, especially in terms of the experience of presence.

This thesis aims to introduce new concepts, theories, and technologies for presence design where the core is actions for creating presence. Thus, the contribution of the thesis can be divided into a technical contribution and a knowledge contribution. Technically, this thesis details novel technologies for improving presence experience during mediated communication (video teleconferencing). The proposed technologies include action robots (including a telepresence mechatronic robot (TEBoT) and a face robot), embodied control techniques (head orientation modeling and virtual reality headset based collaboration), and face reconstruction/retrieval algorithms. The introduced technologies enable action possibilities and embodied interactions that improve the presence experience between the distantly located participants. The novel setups were put into real experimental scenarios, and the well-known social, spatial, and gaze related problems were analyzed.

The developed technologies and the results of the experiments led to the knowledge contribution of this thesis. In terms of knowledge contribution, this thesis presents a more general theoretical conceptual framework for mediated communication technologies. This conceptual framework can guide telepresence researchers toward the development of appropriate technologies for mediated communication applications. Furthermore, this thesis also presents a novel strong concept – presence through actions - that brings in philosophical understandings for developing presence- related technologies. The strong concept - presence through actions is an intermediate-level knowledge that proposes a new way of creating and developing future 'presence artifacts'. Presence- through actions is an action-oriented phenomenological approach to presence that differs from traditional immersive presence approaches that are based (implicitly) on rationalist, internalist views.

Place, publisher, year, edition, pages
Umeå: Umeå universitet, 2017. p. 172
Series
Digital Media Lab, ISSN 1652-6295 ; 22
Keywords
Presence, Immersion, Computer mediated communication, Strong concept, Phenomenology, Philosophy, Biologically inspired system, Neck robot, Head pose estimation, Embodied interaction, Virtual reality headset, Social presence, Spatial presence, Face reconstruction/retrieval, Telepresence system, Quality of interaction, Embodied telepresence system, Mona-Lisa gaze effect, eye-contact
National Category
Electrical Engineering, Electronic Engineering, Information Engineering Computer Systems Communication Systems Signal Processing
Research subject
design; Computer Science; Computer Systems; Computerized Image Analysis
Identifiers
urn:nbn:se:umu:diva-138280 (URN)978-91-7601-730-2 (ISBN)
Public defence
2017-10-10, Triple Helix, Samverkanshuset, Umeå, 09:00 (English)
Opponent
Supervisors
Available from: 2017-08-21 Created: 2017-08-16 Last updated: 2018-06-09Bibliographically approved
Khan, M. S., ur Réhman, S., Söderström, U., Halawani, A. & Li, H. (2016). Face-off: a Face Reconstruction Technique for Virtual Reality (VR) Scenarios. In: Hua G., Jégou H. (Ed.), Computer Vision: ECCV 2016 Workshops. Paper presented at 14th European Conference on Computer Vision, ECCV 2016, Amsterdam, The Netherlands, 8-16 October, 2016 (pp. 490-503). Springer, 9913
Open this publication in new window or tab >>Face-off: a Face Reconstruction Technique for Virtual Reality (VR) Scenarios
Show others...
2016 (English)In: Computer Vision: ECCV 2016 Workshops / [ed] Hua G., Jégou H., Springer, 2016, Vol. 9913, p. 490-503Conference paper, Published paper (Refereed)
Abstract [en]

Virtual Reality (VR) headsets occlude a significant portion of human face. The real human face is required in many VR applications, for example, video teleconferencing. This paper proposes a wearable camera setup-based solution to reconstruct the real face of a person wearing VR headset. Our solution lies in the core of asymmetrical principal component analysis (aPCA). A user-specific training model is built using aPCA with full face, lips and eye region information. During testing phase, lower face region and partial eye information is used to reconstruct the wearer face. Online testing session consists of two phases, i) calibration phase and ii) reconstruction phase. In former, a small calibration step is performed to align test information with training data, while the later uses half face information to reconstruct the full face using aPCA-based trained-data. The proposed approach is validated with qualitative and quantitative analysis.

Place, publisher, year, edition, pages
Springer, 2016
Series
Lecture Notes in Computer Science, ISSN 0302-9743, E-ISSN 1611-3349
Keywords
Virtual Reality, VR headset, Face reconstruction, PCA, wearable setup, Oculus
National Category
Electrical Engineering, Electronic Engineering, Information Engineering Computer Systems Signal Processing
Research subject
Computer Science; Computerized Image Analysis
Identifiers
urn:nbn:se:umu:diva-138277 (URN)10.1007/978-3-319-46604-0_35 (DOI)2-s2.0-84989829345 (Scopus ID)978-3-319-46603-3 (ISBN)978-3-319-46604-0 (ISBN)
Conference
14th European Conference on Computer Vision, ECCV 2016, Amsterdam, The Netherlands, 8-16 October, 2016
Available from: 2017-08-16 Created: 2017-08-16 Last updated: 2018-10-01Bibliographically approved
Khan, M. S., Li, H. & ur Réhman, S. (2016). Gaze perception and awareness in smart devices. International journal of human-computer studies, 92-93, 55-65
Open this publication in new window or tab >>Gaze perception and awareness in smart devices
2016 (English)In: International journal of human-computer studies, ISSN 1071-5819, E-ISSN 1095-9300, Vol. 92-93, p. 55-65Article in journal (Refereed) Published
Abstract [en]

Eye contact and gaze awareness play a significant role for conveying emotions and intentions duringface-to-face conversation. Humans can perceive each other's gaze quite naturally and accurately. However,the gaze awareness/perception are ambiguous during video teleconferencing performed by computer-based devices (such as laptops, tablet, and smart-phones). The reasons for this ambiguity are the(i) camera position relative to the screen and (ii) 2D rendition of 3D human face i.e., the 2D screen isunable to deliver an accurate gaze during video teleconferencing. To solve this problem, researchers haveproposed different hardware setups with complex software algorithms. The most recent solution foraccurate gaze perception employs 3D interfaces, such as 3D screens and 3D face-masks. However, todaycommonly used video teleconferencing devices are smart devices with 2D screens. Therefore, there is aneed to improve gaze awareness/perception in these smart devices. In this work, we have revisited thequestion; how to improve a remote user's gaze awareness among his/her collaborators. Our hypothesis isthat ‘an accurate gaze perception can be achieved by the ‘3D embodiment’ of a remote user's head gestureduring video teleconferencing’. We have prototyped an embodied telepresence system (ETS) for the 3Dembodiment of a remote user's head. Our ETS is based on a 3-DOF neck robot with a mounted smartdevice (tablet PC). The electromechanical platform in combination with a smart device is a novel setupthat is used for studying gaze awareness/perception in 2D screen-based smart devices during videoteleconferencing. Two important gaze-related issues are considered in this work; namely (i) ‘Mona-LisaGaze Effect’ – the gaze is always directed at the person independent of his position in the room, and (ii)‘Gaze Awareness/Faithfulness’ – the ability to perceive an accurate spatial relationship between theobserving person and the object by an actor. Our results confirm that the 3D embodiment of a remoteuser head not only mitigates the Mona Lisa gaze effect but also supports three levels of gaze faithfulness,hence, accurately projecting the human gaze in distant space.

Place, publisher, year, edition, pages
Elsevier, 2016
Keywords
Mona-Lisa gaze effect, gaze awareness, computer-mediated communication, eye contact, head gesture, gaze faithfulness, embodied telepresence system, tablet PC, HCI
National Category
Interaction Technologies Robotics Computer Vision and Robotics (Autonomous Systems)
Identifiers
urn:nbn:se:umu:diva-108568 (URN)10.1016/j.ijhcs.2016.05.002 (DOI)000379367900005 ()
Available from: 2015-09-14 Created: 2015-09-14 Last updated: 2018-06-07Bibliographically approved
Khan, M. S., Li, H. & ur Réhman, S. (2016). Tele-Immersion: Virtual Reality based Collaboration. In: Constantine Stephanidis (Ed.), HCI International 2016: Posters' Extended Abstracts : 18th International Conference, HCI International 2016, Toronto, Canada, July 17-22, 2016, Proceedings, Part I. Paper presented at 18th International Conference on Human-Computer Interaction (HCI International), Toronto, July 17-22, 2016 (pp. 352-357). Springer
Open this publication in new window or tab >>Tele-Immersion: Virtual Reality based Collaboration
2016 (English)In: HCI International 2016: Posters' Extended Abstracts : 18th International Conference, HCI International 2016, Toronto, Canada, July 17-22, 2016, Proceedings, Part I / [ed] Constantine Stephanidis, Springer, 2016, p. 352-357Conference paper, Published paper (Refereed)
Abstract [en]

The 'perception of being present in another space' duringvideo teleconferencing is a challenging task. This work makes an effortto improve upon a user perception of being 'present' in another space byemploying a virtual reality (VR) headset and an embodied telepresencesystem (ETS). In our application scenario, a remote participant usesa VR headset to collaborate with local collaborators. At a local site,an ETS is used as a physical representation of the remote participantamong his/her local collaborators. The head movements of the remoteperson is mapped and presented by the ETS along with audio-video com-munication. Key considerations of complete design are discussed, wheresolutions to challenges related to head tracking, audio-video communi-cation and data communication are presented. The proposed approachis validated by the user study where quantitative analysis is done onimmersion and presence parameters.

Place, publisher, year, edition, pages
Springer, 2016
Series
Communications in Computer and Information Science, ISSN 1865-0929 ; 617
Keywords
Tele-immersion, Virtual reality, Embodied telepresence system, presence, distal attribution, spatial cognition
National Category
Human Computer Interaction Computer Vision and Robotics (Autonomous Systems)
Identifiers
urn:nbn:se:umu:diva-120461 (URN)10.1007/978-3-319-40548-3_59 (DOI)000389727300059 ()978-3-319-40547-6 (ISBN)978-3-319-40548-3 (ISBN)
Conference
18th International Conference on Human-Computer Interaction (HCI International), Toronto, July 17-22, 2016
Available from: 2016-05-16 Created: 2016-05-16 Last updated: 2018-08-10Bibliographically approved
Khan, M. S., Li, H. & ur Réhman, S. (2016). Telepresence Mechatronic Robot (TEBoT): Towards the design and control of socially interactive bio-inspired system. Journal of Intelligent & Fuzzy Systems, 31(5), 2597-2610
Open this publication in new window or tab >>Telepresence Mechatronic Robot (TEBoT): Towards the design and control of socially interactive bio-inspired system
2016 (English)In: Journal of Intelligent & Fuzzy Systems, ISSN 1064-1246, E-ISSN 1875-8967, Vol. 31, no 5, p. 2597-2610Article in journal (Refereed) Published
Abstract [en]

Socially interactive systems are embodied agents that engage in social interactions with humans. From a design perspective, these systems are built by considering a biologically inspired design (Bio-inspired) that can mimic and simulate human-like communication cues and gestures. The design of a bio-inspired system usually consists of (i) studying biological characteristics, (ii) designing a similar biological robot, and (iii) motion planning, that can mimic the biological counterpart. In this article, we present a design, development, control-strategy and verification of our socially interactive bio-inspired robot, namely - Telepresence Mechatronic Robot (TEBoT). The key contribution of our work is an embodiment of a real human-neck movements by, i) designing a mechatronic platform based on the dynamics of a real human neck and ii) capturing the real head movements through our novel single-camera based vision algorithm. Our socially interactive bio-inspired system is based on an intuitive integration-design strategy that combines computer vision based geometric head pose estimation algorithm, model based design (MBD) approach and real-time motion planning techniques. We have conducted an extensive testing to demonstrate effectiveness and robustness of our proposed system.

Keywords
Socially interactive robot, biologically inspired robot, head pose estimation, vision based robot control, model based design, embodied telepresence system
National Category
Robotics Computer Vision and Robotics (Autonomous Systems) Interaction Technologies
Identifiers
urn:nbn:se:umu:diva-108552 (URN)10.3233/JIFS-169100 (DOI)000386532000015 ()
Available from: 2015-09-14 Created: 2015-09-14 Last updated: 2018-06-07Bibliographically approved
Khan, M. S. (2015). Enabling physical action in computer mediated communication: an embodied interaction approach. (Licentiate dissertation). Umeå: Department of Applied Physics and Electronics, Umeå University
Open this publication in new window or tab >>Enabling physical action in computer mediated communication: an embodied interaction approach
2015 (English)Licentiate thesis, comprehensive summary (Other academic)
Place, publisher, year, edition, pages
Umeå: Department of Applied Physics and Electronics, Umeå University, 2015. p. 40
Series
Digital Media Lab, ISSN 1652-6295 ; 20
Keywords
biologically inspired system, parallel robot, neck robot, head pose estimation, embodied interaction, telepresence system, quality of interaction, embodied telepresence system, Mona-Lisa gaze effect, eye-contact
National Category
Communication Systems Interaction Technologies
Identifiers
urn:nbn:se:umu:diva-108569 (URN)978-91-7601-321-2 (ISBN)
Supervisors
Available from: 2015-09-15 Created: 2015-09-14 Last updated: 2018-06-07Bibliographically approved
Khan, M. S. & ur Réhman, S. (2015). Expressive Multimedia: Bringing Action to Physical World by Dancing-Tablet. In: Proceedings of the 2nd Workshop on Computational Models of Social Interactions: Human-Computer-Media Communication. Paper presented at MM '15 ACM Multimedia Conference Brisbane, Australia — October 26 - 30, 2015 (pp. 9-14). ACM Digital Library
Open this publication in new window or tab >>Expressive Multimedia: Bringing Action to Physical World by Dancing-Tablet
2015 (English)In: Proceedings of the 2nd Workshop on Computational Models of Social Interactions: Human-Computer-Media Communication, ACM Digital Library, 2015, p. 9-14Conference paper, Published paper (Refereed)
Abstract [en]

The design practice based on embodied interaction concept focuses on developing new user interfaces for computer devices that merge the digital content with the physical world. In this work we have proposed a novel embodied interaction based design in which the 'action' information of the digital content is presented in the physical world. More specifically, we have mapped the 'action' information of the video content from the digital world into the physical world. The motivating example presented in this paper is our novel dancing-tablet, in which a tablet-PC dances on the rhythm of the song, hence the 'action' information is not just confined into a 2D flat display but also expressed by it. This paper presents i) hardware design of our mechatronic dancing-tablet platform, ii) software algorithm for musical feature extraction and iii) embodied computational model for mapping 'action' information of the musical expression to the mechatronic platform. Our user study shows that the overall perception of audio-video music is enhanced by our dancing-tablet setup.

Place, publisher, year, edition, pages
ACM Digital Library, 2015
Keywords
beat tracking, computational model, dancing robots, embodied interaction, musical interface, tablet-PC}
National Category
Interaction Technologies Media Engineering
Identifiers
urn:nbn:se:umu:diva-120308 (URN)10.1145/2810397.2810399 (DOI)978-1-4503-3747-2 (ISBN)
Conference
MM '15 ACM Multimedia Conference Brisbane, Australia — October 26 - 30, 2015
Available from: 2016-05-15 Created: 2016-05-15 Last updated: 2018-11-28Bibliographically approved
Khan, M. S., ur Réhman, S., La Hera, P., Liu, F. & Li, H. (2014). A pilot user's prospective in mobile robotic telepresence system. In: 2014 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA 2014): . Paper presented at Annual Summit and Conference of Asia-Pacific-Signal-and-Information-Processing-Association (APSIPA), DEC 09-12, 2014, Angkor, CAMBODIA. IEEE
Open this publication in new window or tab >>A pilot user's prospective in mobile robotic telepresence system
Show others...
2014 (English)In: 2014 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA 2014), IEEE, 2014Conference paper, Published paper (Refereed)
Abstract [en]

In this work we present an interactive video conferencing system specifically designed for enhancing the experience of video teleconferencing for a pilot user. We have used an Embodied Telepresence System (ETS) which was previously designed to enhance the experience of video teleconferencing for the collaborators. In this work we have deployed an ETS in a novel scenario to improve the experience of pilot user during distance communication. The ETS is used to adjust the view of the pilot user at the distance location (e.g. distance located conference/meeting). The velocity profile control for the ETS is developed which is implicitly controlled by the head of the pilot user. The experiment was conducted to test whether the view adjustment capability of an ETS increases the collaboration experience of video conferencing for the pilot user or not. The user study was conducted in which participants (pilot users) performed interaction using ETS and with traditional computer based video conferencing tool. Overall, the user study suggests the effectiveness of our approach and hence results in enhancing the experience of video conferencing for the pilot user.

Place, publisher, year, edition, pages
IEEE, 2014
Keywords
Teleconferencing, Collaboration, Computers, Estimation, Noise, Computer science, Human factors
National Category
Robotics Interaction Technologies Computer Vision and Robotics (Autonomous Systems)
Identifiers
urn:nbn:se:umu:diva-108559 (URN)10.1109/APSIPA.2014.7041635 (DOI)000392861900123 ()978-6-1636-1823-8 (ISBN)
Conference
Annual Summit and Conference of Asia-Pacific-Signal-and-Information-Processing-Association (APSIPA), DEC 09-12, 2014, Angkor, CAMBODIA
Available from: 2015-09-14 Created: 2015-09-14 Last updated: 2018-06-07Bibliographically approved
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0002-3037-4244

Search in DiVA

Show all publications