Change search
ReferencesLink to record
Permanent link

Direct link
The importance of balanced data sets for partial least squares discriminant analysis: classification problems using hyperspectral imaging data
Umeå University, Faculty of Science and Technology, Department of Applied Physics and Electronics.
2011 (English)In: Journal of Near Infrared Spectroscopy, ISSN 0967-0335, Vol. 19, no 4, 233-241 p.Article in journal (Refereed) Published
Abstract [en]

This study investigates the effect of imbalanced spectral data in the training set, when developing partial least squares discriminant analysis (PLS-DA) classification models for use in future predictions. The experimental study was performed using a real hyperspectral short-wavelength infrared image data set collected from bakery products (buns) containing contaminants (flies) but similar applications for other insects, paper and plastic were also tested. The contaminants represent a very small proportion of the images relative to the bun. The PLS-DA model aims at accurately detecting and classifying the contaminants and this requires a modification of the calibration data set. The paper deals with problems caused by unbalanced calibration data sets and how to remedy them. In the example it was demonstrated that, by balancing the calibration data from 58,476 bun pixels + 279 fly pixels to 279 bun + 279 fly pixels, the number of true predictions could be improved with a smaller number of PLS components used in the model. The improvement for flies increased from 65% true predictions with ten PLS components to >99% true prediction with five to six PLS components. The true prediction for bun went from 100% to 99.5% with six PLS components which is an acceptable reduction. Theoretical explanations are included.

Place, publisher, year, edition, pages
IM Publications, 2011. Vol. 19, no 4, 233-241 p.
Keyword [en]
hyperspectral imaging, PLS-DA, classification, unbalanced model, obtaining a balanced dataset
National Category
Chemical Sciences
URN: urn:nbn:se:umu:diva-50700DOI: 10.1255/jnirs.932ISI: 000296824700002OAI: diva2:468080
Available from: 2011-12-20 Created: 2011-12-19 Last updated: 2011-12-20Bibliographically approved

Open Access in DiVA

No full text

Other links

Publisher's full text

Search in DiVA

By author/editor
Wiklund Lindström, Susanne
By organisation
Department of Applied Physics and Electronics
In the same journal
Journal of Near Infrared Spectroscopy
Chemical Sciences

Search outside of DiVA

GoogleGoogle Scholar
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Altmetric score

Total: 128 hits
ReferencesLink to record
Permanent link

Direct link