4D thermo-reflectography of cultural heritage. The Codex 4D project: From data acquisition to the implementation in innovative virtual and mixed reality platforms

• 3D thermography from MWIR reflectography images obtained with the same IR camera. • Creation of a 4D model of the manuscript, explorable in real time, including visible and invisible elements. • Integration of the results of chemical and biological investigations into the model. • Semantic annotations of the manuscript 4D model. • Virtual reality environments in the web and engaging holographic showcase for museum. In an effort to try and improve the dissemination of cultural heritage, in this paper a novel method for the four-dimensional (4D) digitization of artefacts, based on images recorded in the mid-wave infrared spectral range, is presented. Such a method is here applied to ancient manuscripts in the frame of the “Codex 4D: journey in four dimensions into the manuscript” project. In the proposed approach, the three-dimensional geometry of the manuscript is reconstructed by processing the reflectographic images, obtained from different points of view with respect to the artefact, through Structure from Motion techniques. Thermograms obtained by means of Pulsed Thermography are also recorded since they provide a depth-resolved characterization of the artefact that is also integrated into the digital reconstruction along the orthogonal direction to the surface, hereafter referred to as the fourth dimension. The results gathered from humanities and scientific studies, are also mapped onto the 4D model in the form of interactive semantic annotations. The goal of such a reconstruction is to allow users to browse the subsurface elements into the 4D model, thus facilitating the study and the exploration of the manuscripts through the inclusion of information about their literary and art-historical contents, materials, execution techniques, and state of conservation. In addition, virtual and mixed reality environments have been developed for different kind of audience such as expert users and museum public. Despite the experimentation presented here was carried out exclusively on manuscripts, in our opinion the methodology can be successfully applied to other types of artefacts.