Pixel fusion and superresolution for Matis handheld thermal imager

Thermal imagers and infrared seekers have the particularity to possess two optical flows, one coming from the observed landscape, the other coming from parasitic radiation of the camera. To this can add a drift of detectors characteristics in time. These parasitic radiation's and detector's characteristics temporal and spatial drifts may have multiple origins: the optics and camera housing thermal drifts, the focal plane array thermal drift during cooldown, the thermal gradients in the focal plane array during cooldown. In case of a very short Joule Thomson cooldown, we are in the hardiest case where all these drifts appear simultaneously. The detector's characteristics, the dark current, the cut-off wavelength, can be very sensitive to the cooling temperature. We can easily understand that, in such conditions non-uniformities corrections are extremely difficult to perform, unless waiting all parameters became stabilized in temperature and then risk to reduce the operational performances. To avoid these difficulties we now suggest, to approach this problem in a fully different way, by fusing information from the successive frames of a video sequence, and by getting benefit of some properties of optical flows of the scene and the camera. This Pixels fusion of successive frames allows to find the motion and vibrations of the carrier. With this knowledge we can extract several new functionalities. One of these is the image stabilization in the terrestrial coordinate system, or the filtering of the carrier vibrations. Another is to derive the non-uniformities corrections of the IR sensor. With this help, we can also implement the detection and tracking of moving targets in the scene. Finally, the improvement of spatial resolution and concurrently the decrease of the temporal noise are also allowed by superresolution. In this paper, we describe the applied methods and present processed videos sequences. New generations of ASICs, FPGAs et DSPs components can now be used to implement powerful digital image processing taking advantage of pixels fusion, even for handheld thermal imagers.