A Survey on Multi-view Learning (1304.5634v1)

Abstract: In recent years, a great many methods of learning from multi-view data by considering the diversity of different views have been proposed. These views may be obtained from multiple sources or different feature subsets. In trying to organize and highlight similarities and differences between the variety of multi-view learning approaches, we review a number of representative multi-view learning algorithms in different areas and classify them into three groups: 1) co-training, 2) multiple kernel learning, and 3) subspace learning. Notably, co-training style algorithms train alternately to maximize the mutual agreement on two distinct views of the data; multiple kernel learning algorithms exploit kernels that naturally correspond to different views and combine kernels either linearly or non-linearly to improve learning performance; and subspace learning algorithms aim to obtain a latent subspace shared by multiple views by assuming that the input views are generated from this latent subspace. Though there is significant variance in the approaches to integrating multiple views to improve learning performance, they mainly exploit either the consensus principle or the complementary principle to ensure the success of multi-view learning. Since accessing multiple views is the fundament of multi-view learning, with the exception of study on learning a model from multiple views, it is also valuable to study how to construct multiple views and how to evaluate these views. Overall, by exploring the consistency and complementary properties of different views, multi-view learning is rendered more effective, more promising, and has better generalization ability than single-view learning.

An Analytical Review of "A Survey on Multi-view Learning"

The paper "A Survey on Multi-view Learning" by Chang Xu, Dacheng Tao, and Chao Xu provides an extensive examination of methodologies for multi-view learning, categorizing them into three primary types: co-training, multiple kernel learning (MKL), and subspace learning. The work not only systematically reviews the principles and assumptions underlying these approaches but also explores view generation and evaluation, presenting experimental comparisons that showcase the efficacy of multi-view learning techniques relative to traditional single-view methods.

Core Principles and Methodologies

Multi-view learning leverages multiple distinct feature sets, or views, which provide a richer representation of the data and can improve model performance. The major principles ensuring the success of multi-view learning algorithms are the consensus and complementary principles. The consensus principle strives to maximize agreement across the views, while the complementary principle seeks to exploit the unique information each view offers.

Co-Training

Co-training, introduced by Blum and Mitchell (1998), hinges on training separate classifiers on distinct views of the data and exchanging the most confidently labeled examples to iteratively enhance their models. This approach has been tailored into various iterations such as co-EM (Nigam and Ghani, 2000), co-regularization (Sindhwani et al., 2005), and graph-based co-training (Yu et al., 2007), among others. These methods generally perform well under assumptions of view independence and sufficiency, although practical work has relaxed these constraints without significantly compromising performance.

Multiple Kernel Learning

MKL optimally combines multiple kernels, each potentially representing a different view of the data. The flexibility of MKL in utilizing a mixture of linear and non-linear combinations of kernels has led to various formulations including semi-definite programming (Laneckriet et al., 2002), quadratic constrained quadratic programming (Bach et al., 2004), and more computationally efficient approaches like simple MKL (Rakotomamonjy et al., 2007). Theoretical bounds for learning kernels emphasize the rich representation capacity provided by MKL while also highlighting the importance of balancing the complexity of the kernel combinations.

Subspace Learning

Subspace learning-based methods, such as Canonical Correlation Analysis (CCA) and its kernelized variant KCCA, aim to identify a shared latent subspace from which multiple views can be generated. This approach reduces the problem of high dimensionality and enables more effective subsequent tasks such as clustering and classification. Advanced techniques that build upon the CCA framework include multi-view Fisher discriminant analysis (Diethe et al., 2008) and shared Gaussian process latent variable models (SGPLVM) (Shon et al., 2006).

View Generation and Evaluation

Generating relevant and diverse views is a critical aspect of multi-view learning. The paper outlines methods for constructing views from data, including feature set partitioning and random subspace methods. Evaluating these views to ensure they are beneficial for learning models involves checking properties like sufficiency and independence, with practical measures addressing noise and redundancy in the views (Christoudias et al., 2008; Liu and Yuen, 2011).

Applications and Empirical Results

Multi-view learning techniques have shown promising results in various domains such as web page classification, image annotation, and object recognition. From the WebKB dataset to multimedia data, multi-view learning models consistently outperform single-view methods. For instance, co-training methods significantly reduce classification error rates in document classification tasks (Blum and Mitchell, 1998; Nigam and Ghani, 2000), while MKL approaches enhance object recognition performance in large-scale image datasets (Varma and Babu, 2009).

Conclusion and Future Directions

The paper underscores the effectiveness of multi-view learning, corroborated through both theoretical insights and empirical validations. However, challenges remain in efficiently constructing multiple views and integrating various algorithms into a general framework. Future directions may involve refining view generation techniques, devising robust evaluation metrics, and enhancing the scalability of multi-view learning algorithms to handle ever-growing datasets.

In conclusion, the paper "A Survey on Multi-view Learning" offers a comprehensive and insightful overview of the field, highlighting the strengths and potential improvements that can be explored to further harness the power of multi-view learning in complex real-world applications.