Photo Forensics | Hany Farid | Freitag, 27.11.2015

Photo Forensics: As Seen on CSI

A digital camera contains a vast array of sensor cells, each with a photo detector and an amplifier. The photo detectors measure incoming light and transform it into an electrical signal. The electrical signals are then converted into pixel values. In an ideal camera, there would be a perfect correlation between the amount of light striking the sensor cells and the pixel values of the digital image. Real devices have imperfections, however, and these imperfections introduce noise in the image.

One source of noise is caused by stray electrons within the sensor cell. These stray electrons combine with the electrons generated by the photo detector as it responds to light. The resulting noise pattern is random, fluctuating from image to image. A second source of noise has very different characteristics. This noise arises from slight variations in the size and material properties of the sensor cells themselves. Physical inconsistencies across the sensor cells lead to differences in the efficiency with which the cells convert light into digital pixel values. These variations, termed photo-response non-uniformity (PRNU), lead to a stable noise pattern that is distinctive to the device.

Although the PRNU will leave a trace in any image, it is easiest to see in a blank image like that of a cloudless sky. If the sky is perfectly uniform, then an ideal camera with an array of identical sensor cells would produce an image that is perfectly homogeneous. In contrast, a real camera will produce an image that has a very faint speckle pattern. The speckle occurs because some sensor cells over-report the amount of incoming light, while others under-report it. If the average sensor cell multiplies the amount of light by a factor of 1, the over-reporting sensor cells multiply it by a factor slightly greater than 1, and the under-reporting cells multiply it by a factor slightly smaller than 1. Unlike sensor noise, which modulates the pixel regardless of its value, PRNU modulates the pixel proportional to its value. Also unlike sensor noise, PRNU is a fixed property of the sensor and so it does not vary from image to image.

The PRNU associated with a particular device is not only stable, it is also distinctive. Even devices of the same make and model have different PRNUs. The stable and distinctive properties of the PRNU allow it to serve two forensic functions. The PRNU can be used to determine whether a particular image is likely to have originated from a given device. The PRNU can also be used to detect localized tampering in an image that was taken from a known device. This second use allows us to confirm the authenticity of an image taken by a photographer who has already produced a body of trusted work.

The estimation of the PRNU relies in part on the distinctive statistical properties of noise, which are unlike the statistical properties of normal image content. (For the details of the computation see 1Jessica Fridrich. Sensor Defects in Digital Image Forensics. In Husrev Taha Sencar and Nasir Memon (eds.), Digital Image Forensics: There is More to a Picture than Meets the Eye (New York: Springer, 2012), 179–218. Jan Lukas, Jessica Fridrich, and Miroslav Goljan. Digital Camera Identification from Sensor Noise. IEEE Transactions on Information Security and Forensics, 1(2):205–214, 2006..) It is possible to get a crude estimate of a device’s PRNU from a single image, but a reliable estimate requires 10 to 20 images (the exact number depends on the quality of the camera, as well as the quality and content of the images).

Example

I estimated the PRNU of my iPhone using 20 images of largely uniform texture-less scenes such as the sky or a blank wall. A small 64 x 64 pixel region of the PRNU is shown below. From the scale at the bottom of this image, you can see that the PRNU factor typically lies between -0.01 to 0.01 (meaning that it only modulates the recorded pixel value by 1%).

I then used my iPhone to photograph a group of students on the Dartmouth College campus. To measure the similarity between this image’s PRNU and the device’s PRNU, I performed a simple pixel correlation. In this case, the resulting correlation value was 0.073. Given that a perfect match would yield a value of 1.0, this correlation may seem to provide insufficient evidence of a match.

(29044)

To test whether a value of 0.073 is sufficient to link the image to my iPhone, I used the photo-sharing website Flickr to randomly select 100,000 images that were the same size as the iPhone image. I then compared the PRNU extracted from each image with the PRNU of the iPhone. The median similarity for these images was 0.0056, more than an order of magnitude smaller than that of the image recorded by the iPhone. Of the 100,000 images I tested, 98.4% had a correlation value less than 0.02, 99.9% had a value less than 0.03, and 99.99% had a value less than 0.04. (The maximum similarity value was 0.0428.) The seemingly low correlation between the iPhone image PRNU and iPhone PRNU was enough to link the image to the device.

Summary

Imperfections in the sensor array introduce stable and distinctive patterns in the images recorded by that device. This photo-response non-uniformity (PRNU) causes some sensor cells to consistently over-report or under-report the amount of measured light. The PRNU pattern for a device can be estimated from a single image or a collection of images. Because the PRNU is both constant and distinctive, it can be used to link an image to a specific device and it can reveal when images from that device have been manipulated.

Further Reading

Jessica Fridrich. Sensor Defects in Digital Image Forensics. In Husrev Taha Sencar and Nasir Memon (eds.), Digital Image Forensics: There is More to a Picture than Meets the Eye (New York: Springer, 2012), 179–218.

Jan Lukas, Jessica Fridrich, and Miroslav Goljan. Digital Camera Identification from Sensor Noise. IEEE Transactions on Information Security and Forensics, 1(2):205–214, 2006.

Portions of this entry are adapted from my upcoming book, Photo Forensics (MIT Press).