Vision and Learning-Based Methods for Scalable and Generalized Image Forensics

Image Forensics has become an important area of research due to the exponential increase in availability and free exchange of media. The accessibility to good quality sensors and image hosting websites is now at the disposal of more users than ever. The increase in sharing of images has also led to more ways to edit image content aimed towards achieving a certain goal. Improved and freely available image editing software can allow a non-technical person to create images with changes undetectable to the naked eye. Whether it is altering one's portrait to look younger, increasing color contrast in natural scenes, or adding external objects to the images to change their perception and understanding. Edits or manipulations with malicious intent can be used to mislead readers or followers. Certain kinds of usage create social and legal concerns with respect to the spread of misinformation while others can skew people's perception of reality. Images manipulated with benign intent behind them may also be fatal to the society by shifting what is perceived as normal and giving the viewers unrealistic expectations.

In order to assess and regulate the quality of media, it is important to devise algorithms that detect and analyze manipulated content in an automated way. This work focuses on solving two such problems - detecting retouching effects in face images and analyzing the provenance of a given image. The former one detects appearance-based manipulations in face images of individuals while the latter implies understanding the evolution history of an image. In addition to understanding single image properties, provenance analysis considers a collection of active variants of the image in question and requires retrieving the stages of evolution of the manipulated media object and the other objects contributing parts to the stages. The proposed methods used to solve these problems focus on efficiency and generalizability as the scale of operation is quite large and the problem is very unconstrained. This work contributes towards formalizing the problem definition and creating solutions that are applicable to general cases of manipulated images at a large scale. We propose methods to (1) improve multi-demographic operation for retouching detection in faces, (2) image provenance analysis appliable to image with diverse content, quality and source. They are evaluated on unconstrained scenarios and tested with large scale datasets. Our proposed pipelines solve important problems in the domain of image forensics and utilize techniques from image analysis, pattern recognition, and computer vision.