Information-Theoretic Analysis of Convolutional Autoencoders: Initial Insights

Frederico Carvalho Fontes do Amaral, Daniel G Silva

DOI: 10.14209/sbrt.2022.1570824859

Evento: XL Simpósio Brasileiro de Telecomunicações e Processamento de Sinais (SBrT2022)

Keywords: Information Theoretic Learning Convolutional Neural Networks Convolutional Autoencoders

Abstract

Despite the success of deep neural networks to solve real world problems, the current theoretical comprehension of their learning mechanisms still deserves further analysis. Recently, various works have explored the use of information-theoretic concepts in order to tackle this issue. This work uses a framework derived from this theory to the study of convolutional autoencoders, in order to better understand its training mechanisms and suggest how the information quantities can be used to determine its bottleneck's size. We conclude by presenting a discussion based on the results obtained that may shed a light on network's learning mechanisms.

Download