A computer vision image differential approach for automatic detection of aggressive behavior in pigs using deep learning

Abstract Pig aggression is a major problem facing the industry as it negatively affects both the welfare and the productivity of group-housed pigs. This study aimed to use a supervised deep learning (DL) approach based on a convolutional neural network (CNN) and image differential to automatically detect aggressive behaviors in pairs of pigs. Different pairs of unfamiliar piglets (N = 32) were placed into one of the two observation pens for 3 d, where they were video recorded each day for 1 h following mixing, resulting in 16 h of video recordings of which 1.25 h were selected for modeling. Four different approaches based on the number of frames skipped (1, 5, or 10 for Diff1, Diff5, and Diff10, respectively) and the amalgamation of multiple image differences into one (blended) were used to create four different datasets. Two CNN models were tested, with architectures based on the Visual Geometry Group (VGG) VGG-16 model architecture, consisting of convolutional layers, max-pooling layers, dense layers, and dropout layers. While both models had similar architectures, the second CNN model included stacked convolutional layers. Nine different sigmoid activation function thresholds between 0.1 and 1.0 were evaluated and a 0.5 threshold was selected to be used for testing. The stacked CNN model correctly predicted aggressive behaviors with the highest testing accuracy (0.79), precision (0.81), recall (0.77), and area under the curve (0.86) values. When analyzing the model recall for behavior subtypes prediction, mounting and mobile non-aggressive behaviors were the hardest to classify (recall = 0.63 and 0.75), while head biting, immobile, and parallel pressing were easy to classify (recall = 0.95, 0.94, and 0.91). Runtimes were also analyzed with the blended dataset, taking four times less time to train and validate than the Diff1, Diff5, and Diff10 datasets. Preprocessing time was reduced by up to 2.3 times in the blended dataset compared to the other datasets and, when combined with testing runtimes, it satisfied the requirements for real-time systems capable of detecting aggressive behavior in pairs of pigs. Overall, these results show that using a CNN and image differential-based deep learning approach can be an effective and computationally efficient technique to automatically detect aggressive behaviors in pigs.