Photo Forensics: J.J. Abrams Style

Bright lights may appear to emanate streaks of light, and this could explain why the sun is often depicted as a circle surrounded by rays. The streaks of light are created by imperfections in the lens of the eye and are referred to as lens flare. Lenses refract (bend) the light rays that enter the pupil of the eye so that the rays focus on the retina. Because of slight imperfections in the lens, some light rays may be scattered or reflected rather than refracted. It is this wayward light that gives rise to lens flare aberrations.

Lens flare may also occur in photographs because of imperfections in the camera lenses. In addition to streaks, lens flare can produce elliptical shapes or starbursts, as shown in the scene below. Although photographers and cinematographers generally avoid lens flare, they may intentionally introduce it for dramatic effect (or possibly overuse it, as in the case of famed film director J.J. Abrams). In this post we will examine how lens flare can be exploited for image forensics.

As described in the previous post, images contain a wealth of information about light source location. People are largely insensitive to this information, however, and so when they edit an image they may unwittingly alter information about the light source location. Several forensic techniques involve checking for consistency in the cues to light source location. The previous post described a technique based on cast shadows. Here we examine a technique based on lens flare.

Lens flare aberrations always lie along lines emanating from the center of the light source. If a scene is illuminated by a single light source (e.g., the sun), then all of the lines delineated by lens flare should intersect at the location of the light source. Any deviations from this common intersection may indicate tampering. In the image below I've outlined three starbursts with yellow circles and drawn a ray through their centers. I’ve also highlighted three streaks with short yellow line segments. These constraint lines all intersect at a single point, as one would expect for an authentic image illuminated by a single light source.

Although our lens flare analysis revealed no inconsistency, we can crosscheck its results with those from other light source analyses. In addition to lens flare, the scene contains cast shadows that can be used to estimate the location of the light source. Recall from the previous post that a point on an object, its cast shadow, and the illuminating light source all lie on a single 3-D line. Because straight lines are imaged as straight lines, this observation also holds in the 2-D image. In the image, a point, its cast shadow, and the light source all lie on a 2-D line. The three cyan-colored lines in the image each connect a point on an object with its cast shadow. These lines, extended upward, intersect at a single point – the projection of the light source.

The two analyses yield contradictory results. Assuming that both analyses were done carefully (see the caveat sections), this is a telltale sign of tampering. This type of inconsistency usually results from the compositing of two images taken under different lighting conditions.

Caveats

A few words of caution when applying this lens flare analysis:

1. The shape of starbursts depends on the camera aperture. I've shown circular starbursts, but starbursts may have a hexagonal shape. Regardless of the shape, the center of the starbursts should lie on a ray emanating from the light source;
2. Lens flare effects can be faint, and this may make it difficult to precisely determine the centers of starbursts or the location of rays. Adjusting the contrast of the image can improve the visibility of lens flare components, making it easier to determine their positions;
3. As with other techniques for analyzing lighting, the analysis of lens flare assumes that a scene is illuminated by a single light source. The assumption of a single light source is reasonable when analyzing outdoor scenes illuminated by the sun. Indoor scenes may contain multiple light sources, however, and care must be taken to differentiate between inconsistent lighting and multiple light sources.

Summary

Photographs may contain lens flare streaks and starbursts, which occur when light from a bright source is scattered and reflected by the camera's lenses. These optical aberrations lie along rays emanating from the light source and so point to the location of this light source. In a typical outdoor scene illuminated by the sun, the various components of the lens flare should be consistent with a single light source location. Moreover, this location should be consistent with location estimates derived from cast shadows. Any inconsistencies in these various estimates of light source location may be evidence of tampering.

Further Reading

Matthias Hullin, Elmar Eisemann, Hans-Peter Seidel, and Sungkil Lee. Physically-Based Real-Time Lens Flare Rendering. ACM Transactions on Graphics, 30(4):108:1–108:10, 2001.

Portions of this entry are adapted from my upcoming book, Photo Forensics (MIT Press).