On The Discriminability of Samples Using Binarized ReLU Activations
Sprache des Titels:
Deutsch
Original Buchtitel:
Proceedings of the 3rd International Conference on Data Science, Machine Learning and Applications (ICDSMLA 2021), 2021
Original Kurzfassung:
Binarized ReLU activations are considered as a metric space equipped
with the Hamming distance. While for two-layer ReLU networks with
random Gaussian weights it can be shown theoretically that local metric properties are approximately preserved, we experimentally study
the discrimination capability in this Hamming space for deeper ReLU
networks and look also at the non-local behaviour. It turns out that
the discrimination capability is approximately preserved as expected.