umu.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Very Low Bitrate Video Communication: A Principal Component Analysis Approach
Umeå University, Faculty of Science and Technology, Department of Applied Physics and Electronics.
2008 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

A large amount of the information in conversations come from non-verbal cues such as facial expressions and body gesture. These cues are lost when we don't communicate face-to-face. But face-to-face communication doesn't have to happen in person. With video communication we can at least deliver information about the facial mimic and some gestures. This thesis is about video communication over distances; communication that can be available over networks with low capacity since the bitrate needed for video communication is low.

A visual image needs to have high quality and resolution to be semantically meaningful for communication. To deliver such video over networks require that the video is compressed. The standard way to compress video images, used by H.264 and MPEG-4, is to divide the image into blocks and represent each block with mathematical waveforms; usually frequency features. These mathematical waveforms are quite good at representing any kind of video since they do not resemble anything; they are just frequency features. But since they are completely arbitrary they cannot compress video enough to enable use over networks with limited capacity, such as GSM and GPRS.

Another issue is that such codecs have a high complexity because of the redundancy removal with positional shift of the blocks. High complexity and bitrate means that a device has to consume a large amount of energy for encoding, decoding and transmission of such video; with energy being a very important factor for battery-driven devices.

Drawbacks of standard video coding mean that it isn't possible to deliver video anywhere and anytime when it is compressed with such codecs. To resolve these issues we have developed a totally new type of video coding. Instead of using mathematical waveforms for representation we use faces to represent faces. This makes the compression much more efficient than if waveforms are used even though the faces are person-dependent.

By building a model of the changes in the face, the facial mimic, this model can be used to encode the images. The model consists of representative facial images and we use a powerful mathematical tool to extract this model; namely principal component analysis (PCA). This coding has very low complexity since encoding and decoding only consist of multiplication operations. The faces are treated as single encoding entities and all operations are performed on full images; no block processing is needed. These features mean that PCA coding can deliver high quality video at very low bitrates with low complexity for encoding and decoding.

With the use of asymmetrical PCA (aPCA) it is possible to use only semantically important areas for encoding while decoding full frames or a different part of the frames.

We show that a codec based on PCA can compress facial video to a bitrate below 5 kbps and still provide high quality. This bitrate can be delivered on a GSM network. We also show the possibility of extending PCA coding to encoding of high definition video.

Place, publisher, year, edition, pages
Umeå: Tillämpad fysik och elektronik , 2008. , 90 p.
Series
Digital Media Lab, ISSN 1652-6295 ; 11
Keyword [en]
Video compression, Very low bitrate, Principal component analysis, Complexity, Semantically important areas, Wearable video
National Category
Other Electrical Engineering, Electronic Engineering, Information Engineering
Identifiers
URN: urn:nbn:se:umu:diva-1808ISBN: 978-91-7264-644-5 (print)OAI: oai:DiVA.org:umu-1808DiVA: diva2:142053
Public defence
2008-09-26, N200, Naturvetarhuset, Umeå universitet, Umeå, 10:00 (English)
Opponent
Supervisors
Available from: 2008-09-05 Created: 2008-09-05 Last updated: 2010-11-05Bibliographically approved
List of papers
1. Full-frame video coding for facial video sequences based on principal component analysis
Open this publication in new window or tab >>Full-frame video coding for facial video sequences based on principal component analysis
2005 (English)In: Proceedings of Irish Machine Vision and Image Processing Conference, 25-32 p.Article in journal (Refereed) Published
Identifiers
urn:nbn:se:umu:diva-3375 (URN)
Available from: 2008-09-05 Created: 2008-09-05 Last updated: 2016-02-23Bibliographically approved
2. Representation bound for human facial mimic with the aid of principal component analysis
Open this publication in new window or tab >>Representation bound for human facial mimic with the aid of principal component analysis
2010 (English)In: International Journal of Image and Graphics, ISSN 0219-4678, Vol. 10, no 3, 343-363 p.Article in journal (Refereed) Published
Abstract [en]

In this paper, we examine how much information is needed to represent the facial mimic, based on Paul Ekman's assumption that the facial mimic can be represented with a few basic emotions. Principal component analysis is used to compact the important facial expressions. Theoretical bounds for facial mimic representation are presented both for using a certain number of principal components and a certain number of bits. When 10 principal components are used to reconstruct color image video at a resolution of 240 × 176 pixels the representation bound is on average 36.8 dB, measured in peak signal-to-noise ratio. Practical confirmation of the theoretical bounds is demonstrated. Quantization of projection coefficients affects the representation, but a quantization with approximately 7-8 bits is found to match an exact representation, measured in mean square error.

Place, publisher, year, edition, pages
World Scientific Publishing Company, 2010
Keyword
Distortion bound, rate-distortion bound, facial mimic, basic emotions, principal component analysis, PCA
National Category
Media Engineering
Identifiers
urn:nbn:se:umu:diva-5428 (URN)10.1142/S0219467810003810 (DOI)
Available from: 2011-09-19 Created: 2006-10-13 Last updated: 2017-12-14Bibliographically approved
3. Eigenspace compression for very low bitrate transmission of facial video
Open this publication in new window or tab >>Eigenspace compression for very low bitrate transmission of facial video
2007 (English)In: IASTED International conference on Signal Processing, Pattern Recognition and ApplicationsArticle in journal (Refereed) Published
Identifiers
urn:nbn:se:umu:diva-3377 (URN)
Available from: 2008-09-05 Created: 2008-09-05 Last updated: 2010-01-26Bibliographically approved
4. Ultra low bit-rate video communication: video coding = pattern recognition
Open this publication in new window or tab >>Ultra low bit-rate video communication: video coding = pattern recognition
2006 (English)In: Proceedings of the 25th Picture Coding Symposium, 2006Conference paper, Published paper (Refereed)
Identifiers
urn:nbn:se:umu:diva-22982 (URN)
Available from: 2009-05-25 Created: 2009-05-25 Last updated: 2010-01-21Bibliographically approved
5. Asymmetrical Principal Component Analysis: Theory and Its Applications to Facial Video Coding
Open this publication in new window or tab >>Asymmetrical Principal Component Analysis: Theory and Its Applications to Facial Video Coding
2011 (English)In: Effective Video Coding for Multimedia Applications / [ed] Sudhakar Radhakrishnan, InTech Open , 2011, 95-110 p.Chapter in book (Refereed)
Place, publisher, year, edition, pages
InTech Open, 2011
National Category
Media Engineering
Identifiers
urn:nbn:se:umu:diva-3379 (URN)
Available from: 2008-09-05 Created: 2008-09-05 Last updated: 2012-02-17Bibliographically approved
6. Side view driven facial video coding
Open this publication in new window or tab >>Side view driven facial video coding
(English)Article in journal (Refereed) Submitted
Identifiers
urn:nbn:se:umu:diva-3380 (URN)
Available from: 2008-09-05 Created: 2008-09-05 Last updated: 2016-02-23Bibliographically approved
7. High definition wearable video communication
Open this publication in new window or tab >>High definition wearable video communication
2009 (English)In: Image analysis: 16th Scandinavian Conference, SCIA 2009, Oslo, Norway, June 15-18, 2009. Proceedings / [ed] Arnt-Børre Salberg, Jon Yngve Hardeberg, Robert Jenssen, Heidelberg: Springer Berlin , 2009, 500-512 p.Conference paper, Published paper (Refereed)
Abstract [en]

High definition (HD) video can provide video communication which is as crisp and sharp as face-to-face communication. Wearable video equipment also provide the user with mobility; the freedom to move. HD video requires high bandwidth and yields high encoding and decoding complexity when encoding based on DCT and motion estimation is used. We propose a solution that can drastically lower the bandwidth and complexity for video transmission. Asymmetrical principal component analysis can initially encode HD video into bitrates which are low considering the type of video (< 300 kbps) and after a startup phase the bitrate can be reduced to less than 5 kbps. The complexity for encoding and decoding of this video is very low; something that will save battery power for mobile devices. All of this is done only at the cost of lower quality in frame areas which aren’t considered semantically important.

Place, publisher, year, edition, pages
Heidelberg: Springer Berlin, 2009
Series
Lecture Notes in Computer Science, ISSN 1611-3349 ; 5575
National Category
Physical Sciences
Identifiers
urn:nbn:se:umu:diva-35809 (URN)47833q8x9651j834/ (DOI)978-3-642-02229-6 (ISBN)
Conference
Scandinavian Conference on Image Analysis, SCIA
Available from: 2010-09-06 Created: 2010-09-06 Last updated: 2012-06-05Bibliographically approved

Open Access in DiVA

fulltext(1507 kB)917 downloads
File information
File name FULLTEXT01.pdfFile size 1507 kBChecksum SHA-1
5471a3ff241b6086381235496343f7170d38df3d6832a243895ec897afe3e5947062d953
Type fulltextMimetype application/pdf

Search in DiVA

By author/editor
Söderström, Ulrik
By organisation
Department of Applied Physics and Electronics
Other Electrical Engineering, Electronic Engineering, Information Engineering

Search outside of DiVA

GoogleGoogle Scholar
Total: 917 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 1622 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf