Deep Within-Class Covariance Analysis for Robust Deep Audio Representation Learning
Sprache des Vortragstitels:
NeurIPS 2018 Interpretability and Robustness for Audio, Speech and Language Workshop
Sprache des Tagungstitel:
Deep Neural Networks (DNNs) are known for excellent performance in supervised tasks such as classification. Convolutional Neural Networks (CNNs), in particular, can learn effective features and build high-level representations that can be used for classification, but also for querying and nearest neighbor search.
However, CNNs have also been shown to suffer from a performance drop when the distribution of the data changes from training to test data. In this paper we analyze the internal representations of CNNs and observe that the representations of unseen data in each class, spread more (with higher variance) in the embedding space of the CNN compared to representations of the training data.
More importantly, this difference is more extreme if the unseen data comes from a shifted distribution. Based on this observation, we objectively evaluate the degree of representation?s variance in each class by applying eigenvalue decomposition on the within-class covariance of the internal representations of CNNs and observe the same behaviour. This can be problematic as larger variances might lead to mis-classification if the sample crosses the decision boundary of its class. We apply nearest neighbor classification on the representations and empirically show that the embeddings with the high variance actually have significantly worse KNN classification performances, although this could not be foreseen from their end-to-end classification results. To tackle this problem, we propose Deep Within-Class Covariance Analysis (DWCCA), a deep neural network layer that significantly reduces the within-class covariance of a DNN?s representation, improving performance on unseen test data from a shifted distribution. We demonstrate that not only does DWCCA significantly improve the network?s internal representation, it also increases the end-to-end classification accuracy, especially when the test set exhibits a slight distribution shift.