Photo Forensics: Can You Enhance That?

When you view a sign from a highly oblique angle, the perspective distortion may cause the text and graphics to be unrecognizable. To make the sign interpretable, you could change your vantage point and view the sign head-on. But what if the sign appears at an oblique angle in a photograph? Is it possible to remove the perspective distortion and view the sign head-on? This question arises in forensic settings when the evidence contained within an image is obscured by perspective distortion. In this post, I describe how perspective distortion can be removed to reveal, for example, the identity of a barely visible license plate.

Perspective distortion arises when the parts of an object are located at different distances from the viewer. The parts that are distant will appear at a smaller scale than the parts that are near. To render the distorted object from a more advantageous view, it is necessary to compensate for this distance scaling, and this requires knowing the 3-D shape of the object. For most objects in the world, the 3-D shape is unknown and unrecoverable from a single image. A commonly occurring exception are rectangular planar surfaces.

The process of removing perspective distortion from such planar surfaces is called planar rectification, and it can be accomplished in Photoshop in a few simple steps (these directions are specified using version CS6):

1. Load the image into Photoshop
2. Select Filter —> Vanishing Point.
3. Select Create Plane Tool from the toolbar on the left.
   
4. Select the four corners of the perspectively distorted rectangular surface in the image.
5. Photoshop will superimpose a gridded plane atop your image (shown below). This is the 3-D model of the plane required to remove perspective distortion.
6. Select Return 3D Layer to Photoshop from the Settings and Commands menu.
   
7. Select OK.
8. A new layer will be created that contains a 3-D model of the planar surface (shown below). Double click on the Diffuse component of the texture layer and the undistorted version of your image will appear in a new window.

Example

In the image below, the perspective distortion is so severe that the identity of the license plate seems lost. Nonetheless, using the series of steps described above, I was able to make the characters clearly legible.

The rectification is obviously not perfect: (1) the top and bottom of the license plate are not visible because the car bumper occludes the top and the plate is bent along the bottom; (2) the sides of the license plate are not straight because the plate is bent at the edges; and (3) the rusted bolts along the top are highly distorted because the bolts protrude from the surface. Each of these distortions arises for the same reason – the license plate is not perfectly flat. It is a practical reality that most surfaces are not perfectly flat, and so this type of residual distortion is fairly common. Despite the imperfect rectification, the removal of the perspective distortion is still quite effective at revealing the identity of the license plate.

Summary

Perspective distortion may obscure critical information in an image. If the critical information lies on a rectangular plane, it is possible to remove the perspective distortion through a process called planar rectification. Photoshop provides the basic tools for accomplishing this rectification.

Further Reading

Richard Hartley and Andrew Zisserman. Multiple View Geometry in Computer Vision (Cambridge: Cambridge University Press, 2004).

David Liebowitz and Andrew Zisserman. Metric Rectification for Perspective Images of Planes. IEEE Conference on Computer Vision and Pattern Recognition, 482–488, 1998.

Portions of this entry are adapted from my upcoming book, Photo Forensics (MIT Press).