A Robust Temporal Alignment Technique for Infrared and Visible-Light Video Sequences

Traditional temporal video alignment techniques depend on the existence of a sufficient number of mutual scene characteristics to perform temporal synchronization. However, for video sequences originating from different spectral modalities such information may fail to exist, subsequently leading to unreliable alignment results. In this work, a novel, practical approach for the temporal alignment of infrared (IR) and visiblelight video sequences is introduced. Utilizing a planar alignment device consisting of an array of miniature light bulbs, we propose to increase the number of mutual feature points within the frames of a temporally misaligned IR/visible-light video sequence pair. Thereby, we provide an abundant number of mutual scene characteristics which can subsequently be used to determine the time-shift between the two sequences. Tests with five different IR/visible-light sequence pairs suggest that our technique is able to estimate the temporal offset with a very high confidence level. Furthermore, due to the simplicity of the construction of the alignment board, the proposed framework can be seamlessly integrated within a stereo camera calibration scheme.