Analysis of error landscapes in multi-layered neural networks for classification

Artificial neural networks are inherently high-dimensional, which limits our ability to visualise and understand their inner workings. Neural network architecture and training algorithm parameters are usually optimised on an ad hoc basis, with very limited insight into the nature of the objective function landscape. This study proposes using fitness landscape analysis to quantify topological properties of neural network error landscapes. Five techniques from the fitness landscape analysis field are adapted to work with neural network error landscapes. These techniques are then used to analyse how the error landscape changes under different error measurements and different number of hidden layers. The results show that fitness landscape analysis provides valuable insight into neural network error landscapes, and could be used for architecture selection.