Feed-Forward Neural Networks

Feed-forward neural networks were the earliest implementations within deep learning. These networks are called feed-forward because the information within them moves only in one direction (forward)—that is, from the input nodes (units) towards the output units. In this chapter, we will cover some key concepts around feed-forward neural networks that serve as a foundation for various topics within deep learning. We will start by looking at the structure of a neural network, followed by how they are trained and used for making predictions. We will also take a brief look at the loss functions that should be used in different settings, the activation functions used within a neuron, and the different types of optimizers that could be used for training. Finally, we will stitch together each of these smaller components into a full-fledged feed-forward neural network with PyTorch.