Visual Psychophysics for Computer Vision

By providing substantial amounts of data and standardized evaluation protocols, datasets in computer vision have helped fuel advances across all areas of visual recognition. But even in light of breakthrough results on recent benchmarks, it is still fair to ask if our recognition algorithms are doing as well as we think they are. The vision sciences at large make use of a very different evaluation regime known as Visual Psychophysics to study visual perception. Psychophysics is the quantitative examination of the relationships between controlled stimuli and the behavioral responses they elicit in experimental test subjects. Instead of using summary statistics to gauge performance, psychophysics directs us to construct item-response curves made up of individual stimulus responses to find perceptual thresholds, thus allowing one to identify the point at which a subject can no longer reliably recognize the stimulus.

In this work I will demonstrate the efficacy of visual psychophysics for computer vision from four main perspectives: visual psychophysics for evaluation, visual psychophysics for explainability, visual psychophysics for improved generalizability, and finally it presents a real life application that could substantially benefit from the capabilities visual psychophysics has to offer. The arch from evaluation to application completes a life cycle of a computer vision algorithm, providing positive evidence to the community that visual psychophysics can aid any stage of development.