The goal with this project was to improve an existing face recognition system for video streams by using adaptive object tracking to track faces between frames. The knowledge of what faces occur and do not occur in subsequent frames was used to filter false faces and to better identify real ones.
The recognition ability was tested by measuring how many faces were found and how many of them were correctly identified in two short video files. The tests also looked at the number of false face detections. The results were compared to a reference implementation that did not use object tracking.
Two identification modes were tested: the default and strict modes. In the default mode, whichever person is most similar to a given image patch is accepted as the answer. In strict mode, the similarity also has to be above a certain threshold.
The first video file had a fairly high image quality. It had only frontal faces, one at a time. The second video file had a slightly lower image quality. It had up to two faces at a time, in a larger variety of angles. The second video was therefore a more difficult case. The results show that the number of detected faces increased by 6-21% in the two video files, for both identification modes, compared to the reference implementation.
In the meantime, the number of false detections remained low. In the first video file, there were fewer than 0.009 false detections per frame. In the second video file, there were fewer than 0.08 false detections per frame.
The number of faces that were correctly identified increased by 8-22% in the two video files in default mode. In the first video file, there was also a large improvement in strict mode, as it went from recognising 13% to 85% of all faces. In the second video file, however,neither implementation managed to identify anyone in strict mode.
The conclusion is that object tracking is a good tool for improving the accuracy of face recognition in video streams. Anyone implementing face recognition for video streams should consider using object tracking as a central component.