Poses of People in Art: A Data Set for Human Pose Estimation in Digital Art History

With the Poses of People in Art data set, we introduce the first openly licensed data set for estimating human poses in art and validating human pose estimators. It consists of 2,454 images from 22 art-historical depiction styles, including those that have increasingly moved away from lifelike representations of the body since the 19 th century. A total of 10,749 human figures are enclosed by rectangular bounding boxes, with a maximum of four per image labeled by up to 17 keypoints. For machine learning purposes, the data set is divided into three subsets—training, validation, and test—that follow the JSON-based Microsoft Common Objects in Context (COCO) format, respectively. Each image annotation provides metadata from the online visual art encyclopedia WikiArt, in addition to mandatory fields. In this paper, we report on the acquisition and constitution of the data set, address various application scenarios, and discuss the prospects for a digitally supported art history. We show that the data set allows for the study of body phenomena in art, whether on the level of individual figures, which can thus be captured in their subtleties, or entire figure constellations, whose position or distance to each other is considered.