Discriminative Learning and Recognition of Image Set Classes Using Canonical Correlations

• Published in: IEEE transactions on pattern analysis and machine intelligence 2007-06, Vol.29 (6), p.1005-1018
• Main Authors: Tae-Kyun Kim, Kittler, J., Cipolla, R.
• Format: Article
• Language: English
• Subjects: Algorithms; Applied sciences; Artificial Intelligence; Cameras; canonical correlation; canonical correlation analysis; Cluster Analysis; Computer science; control theory; systems; Computer vision; Correlation; Correlation analysis; Discriminant Analysis; Exact sciences and technology; Face recognition; Illumination; Image Enhancement - methods; Image Interpretation, Computer-Assisted - methods; Image recognition; image sets; Information Storage and Retrieval - methods; Learning; Lighting; Linear discriminant analysis; Matching; Methods; Neural networks; Numerical Analysis, Computer-Assisted; Object recognition; orthogonal subspace method; Pattern Recognition, Automated - methods; Pattern recognition. Digital image processing. Computational geometry; principal angles; Recognition; Reproducibility of Results; Robustness; Sensitivity and Specificity; Signal Processing, Computer-Assisted; Statistics as Topic; Studies; Subtraction Technique; Vectors
• ISSN: 0162-8828; 1939-3539; 2160-9292
• DOI: 10.1109/TPAMI.2007.1037

We address the problem of comparing sets of images for object recognition, where the sets may represent variations in an object's appearance due to changing camera pose and lighting conditions. canonical correlations (also known as principal or canonical angles), which can be thought of as the angles between two d-dimensional subspaces, have recently attracted attention for image set matching. Canonical correlations offer many benefits in accuracy, efficiency, and robustness compared to the two main classical methods: parametric distribution-based and nonparametric sample-based matching of sets. Here, this is first demonstrated experimentally for reasonably sized data sets using existing methods exploiting canonical correlations. Motivated by their proven effectiveness, a novel discriminative learning method over sets is proposed for set classification. Specifically, inspired by classical linear discriminant analysis (LDA), we develop a linear discriminant function that maximizes the canonical correlations of within-class sets and minimizes the canonical correlations of between-class sets. Image sets transformed by the discriminant function are then compared by the canonical correlations. Classical orthogonal subspace method (OSM) is also investigated for the similar purpose and compared with the proposed method. The proposed method is evaluated on various object recognition problems using face image sets with arbitrary motion captured under different illuminations and image sets of 500 general objects taken at different views. The method is also applied to object category recognition using ETH-80 database. The proposed method is shown to outperform the state-of-the-art methods in terms of accuracy and efficiency