Umeå University's logo

This chapter describes the transformation of a young physical organic chemist (SW, 1964), from a believer in first principles models to a middle-aged chemometrician (SW, 1974) promoting empirical and semiempirical "data driven, soft, analogy" models for the design of experiments and the analysis of the resulting data. This transformation was marked by a number of influential events, each tipping the balance towards the data driven, soft, analogy models until the point of no return in 1974. On June 10, 1974, Bruce and I together with our research groups joined forces formed the Chemometrics Society (later renamed to the International Chemometrics Society), and we took off into multidimensional space. This review of my personal scientific history, inspired and encouraged by Bruce, is illustrated by examples of method development driven by necessity to solve specific problems and leading to data driven soft models, which, at least in my own eyes, were superior to the classical first principles approaches to the same problems. Bruce and I met at numerous conferences between 1975 and 1990, but after that, Bruce and I gradually slid out of the academic world, and now Bruce has taken his final step.

A personal view is given about the gradual development of projection methods-also called bilinear, latent variable, and more-and their use in chemometrics. We start with the principal components analysis (PCA) being the basis for more elaborate methods for more complex problems such as soft independent modeling of class analogy, partial least squares (PLS), hierarchical PCA and PLS, PLS-discriminant analysis, Orthogonal projection to latent structures (OPLS), OPLS-discriminant analysis and more. From its start around 1970, this development was strongly influenced by Bruce Kowalski and his group in Seattle, and his realization that the multidimensional data profiles emerging from spectrometers, chromatographs, and other electronic instruments, contained interesting information that was not recognized by the current one variable at a time approaches to chemical data analysis. This led to the adoption of what in statistics is called the data analytical approach, often called also the data driven approach, soft modeling, and more. This approach combined with PCA and later PLS, turned out to work very well in the analysis of chemical data. This because of the close correspondence between, on the one hand, the matrix decomposition at the heart of PCA and PLS and, on the other hand, the analogy concept on which so much of chemical theory and experimentation are based. This extends to numerical and conceptual stability and good approximation properties of these models. The development is informally summarized and described and illustrated by a few examples and anecdotes.

The amount of data measured during a typical manufacturing process is immense. To efficiently utilize these data without becoming overwhelmed with confusing and often conflicting information is difficult to impossible when using traditional univariate methods. Multivariate data mining methods can be used to examine large data sets by extracting relationships between variables to highlight variable correlations and deviations. Specifically, PLS-trees can be used to quickly identify significant clusters in large datasets and to highlight the differences within the groups.

We introduce a new measure for the importance of predictor variables, X, for the separation of two groups (classes) of observations. The measure is a Graphical Index of Separation (GIOS), and is, for each predictor, determined from the distribution of all possible pairs of observations with one from each group. GIOS is quantitative, intuitively simple and easy to interpret. The GIOS is straightforward to visualize in bivariate plots, and line or bar plots for larger number of variables. The approach applies both to discriminant analyses such as LDA, SIMCA, PLS-DA, OPLS-DA and to quantitative modeling such as MLR, PLS and OPLS. In the latter case, the observations are first divided into two groups based on their response values, Y. The GIOS approach is illustrated by PLS-DA/OPLS-DA and SIMCA-classification of a number of multivariate data sets with few and many variables relative to the number of observations.

Batch-wise manufacturing (applied in fermentation, cell culturing, and chemical synthesis), gives rise to three-way data arrays when several process variables are measured on the process at regular intervals. The same data structure results from in pharmacokinetics and metabonomics, when data profiles of are taken from individuals at specified intervals.

The modeling approaches of three-way batch data for the purpose of understanding, fault detection, control, and prediction, fall in two broad categories,

1. using summarizing variables such as discrete features from the trajectories – landmark points (e.g., peak temp., slopes, times in various phases, etc.), and then forming a batchwise X matrix from these and analyzing by regular PCA/PLS;

2. unfolding the three way array of batch data into a two way matrix (can be done in several ways), followed by PCA/PLS of the two way array to extract an efficient feature set – i.e. latent variables.

The established approaches of batch data analysis are reviewed and illustrated by three examples, of yeast production, nylon manufacturing, and of a drying process step.

Abstract: A method and system for partitioning (clustering) large amounts of data in a relatively short processing time. The method involves providing a first data matrix and a second data matrix where each of the first and second data matrices includes one or more variables, and a plurality of data points. The method also involves determining a first score from the first data matrix using a partial least squares (PLS) analysis or orthogonal PLS (OPLS) analysis and partitioning the first and second data matrices (e.g., row-wise) into a first group and a second group based on the sorted first score, the variance of the first data matrix, and a variance of the first and second groups relative to the variances of the first and second data matrices.

A hierarchical clustering approach based on a set of PLS models is presented. Called PLS-Trees (R), this approach is analogous to classification and regression trees (CART), but uses the scores of PLS regression models as the basis for splitting the clusters, instead of the individual X-variables. The split of one cluster into two is made along the sorted first X-score (t(1)) of a PLS model of the cluster, but may potentially be made along a direction corresponding to a combination of scores. The position of the split is selected according to the improvement of a weighted combination of (a) the variance of the X-score, (b) the variance of Y and (c) a penalty function discouraging an unbalanced split with very different numbers of observations. Cross-validation is used to terminate the branches of the tree, and to determine the number of components of each cluster PLS model. Some obvious extensions of the approach to OPLS-Trees and trees based on hierarchical PLS or OPLS models with the variables divided in blocks depending on their type, are also mentioned. The possibility to greatly reduce the number of variables in each PLS model on the basis of their PLS w-coefficients is also pointed out. The approach is illustrated by means of three examples. The first two examples are quantitative structure-activity relationship (QSAR) data sets, while the third is based on hyperspectral images of liver tissue for identifying different sources of variability in the liver samples.

This report describes significance testing for PLS and OPLS® (orthogonal PLS) models. The testing is applicable to single-Y cases and is based on ANOVA of the cross-validated residuals (CV-ANOVA). Two variants of the CV-ANOVA are introduced. The first is based on the cross-validated predictive residuals of the PLS or OPLS model while the second works with the cross-validated predictive score values of the OPLS model. The two CV-ANOVA diagnostics are shown to work well in those cases where PLS and OPLS work well, that is, for data with many and correlated variables, missing data, etc. The utility of the CV-ANOVA diagnostic is demonstrated using three datasets related to (i) the monitoring of an industrial de-inking process; (ii) a pharmaceutical QSAR problem and (iii) a multivariate calibration application from a sugar refinery. Copyright © 2008 John Wiley & Sons, Ltd.

Pell, Ramos and Manne (PRM) in a recent article in this journal claim that the conventional PLS algorithm with orthogonal scores has an inherent inconsistency in that it uses different model spaces for calculating the prediction model coefficients and for calculating the X-space model and it's residuals [1]. We disagree with PRM. All PLS model scores, residuals, coefficients, etc., obtained by the conventional PLS algorithm do come from the same underlying latent variable (LV) model, and not from different models or model spaces as PRM suggest. PRM have simply posed a different model with different assumptions and obtained slightly different results, as should have been expected.

During the last two decades, a number of methods have been developed and evaluated for selecting the optimal number of components in a PLS model. In this paper, a new method is introduced that is based on a randomization test. The advantage of using a randomization test is that in contrast to cross validation (CV), it requires no exclusion of data, thus avoiding problems related to data exclusion, for example in designed experiments. The method is tested using simulated data sets for which the true dimensionality is clearly defined and also compared to regularly used methods for 10 real data sets. The randomization test works as a good statistical selection tool in combination with other selection rules. It also works as an indicator when the data require a pre-treatment.