Information-Theoretic Analysis of Convolutional Autoencoders: Initial Insights

Despite the success of deep neural networks to solve real world problems, the current theoretical comprehension of their learning mechanisms still deserves further analysis. Recently, various works have explored the use of information-theoretic concepts in order to tackle this issue. This work uses a framework derived from this theory to the study of convolutional autoencoders, in order to better understand its training mechanisms and suggest how the information quantities can be used to determine its bottleneck's size. We conclude by presenting a discussion based on the results obtained that may shed a light on network's learning mechanisms.