Matthias Dorfer, Rainer Kelz, Gerhard Widmer,
"Deep Linear Discriminant Analysis"
: Proceedings of the International Conference on Learning Representations (ICLR), 5-2016
Original Titel:
Deep Linear Discriminant Analysis
Sprache des Titels:
Englisch
Original Buchtitel:
Proceedings of the International Conference on Learning Representations (ICLR)
Original Kurzfassung:
We introduce Deep Linear Discriminant Analysis (
DeepLDA
) which learns lin-
early separable latent representations in an end-to-end fashion. Classic LDA ex-
tracts features which preserve class separability and is used for dimensionality
reduction for many classification problems. The central idea of this paper is to
put LDA on top of a deep neural network. This can be seen as a non-linear ex-
tension of classic LDA. Instead of maximizing the likelihood of target labels for
individual samples, we propose an objective function that pushes the network to
produce feature distributions which: (a) have low variance within the same class
and (b) high variance between different classes. Our objective is derived from the
general LDA eigenvalue problem and still allows to train with stochastic gradient
descent and back-propagation. For evaluation we test our approach on three dif-
ferent benchmark datasets (MNIST, CIFAR-10 and STL-10). DeepLDA produces
competitive results on MNIST and CIFAR-10 and outperforms a network trained
with categorical cross entropy (having the same architecture) on a supervised set-
ting of STL-10.
Forschungs-