The Hardest Subject in Visual Inspection? A Mirror. Seeing Scratches and Flatness on a Reflective Surface with TR-300
kotai2003What is the toughest subject in visual inspection? It is something most people look into every morning: a mirror. First, take a look at the image below.
A mirror hanging on an office wall, clearly reflecting the whole room: the window, blinds, garden greenery, the photographer, a desk, and even a monitor showing TOMOMI RESEARCH's screen
This is an entirely ordinary mirror hanging on the wall of our office. What it shows is the greenery outside the window, the blinds, a desk, a monitor displaying TOMOMI RESEARCH software, and the person holding up a smartphone to take the photo. In other words, everything in the picture is "the room," not the mirror itself. Is the mirror's surface flat? Is it scratched? Is it wavy? The thing that matters, "the surface itself," is completely hidden behind these sharp reflections.
This is the limit of the naked eye and an ordinary camera. A mirror bounces light straight back, so all you can capture is "something reflected in the mirror." So what does the mirror's "surface" actually look like? In this post, we present the results of observing a mirror surface, which could be called the "final boss" of visual inspection, with our AI visual inspection system TR-300. How does TR-300 extract "the shape of the surface" from a mirror in which the camera should see nothing of the surface? Follow along with actual images.
Seen from the camera, a mirror is nothing but reflections
Next, here is the same mirror set on the TR-300 imaging stage.
The mirror set on the TR-300 imaging stage. The equipment frame and the view from the window are reflected in the mirror surface
Here too, the black frame of the equipment and the scenery coming in through the window are clearly reflected on the mirror. Just as when it hung on the office wall, nothing it shows is "the mirror itself." What we are seeing is merely the surroundings bounced back by the mirror. Whether the mirror's surface is flat, scratched, or wavy, the condition of the surface itself is still completely hidden behind the reflections.
This is why mirrors are the hardest challenge in visual inspection.
Why are mirrors so difficult?
A highly reflective surface like a mirror is technically called a specular surface (a surface with mirror-like reflection). A specular surface reflects incoming light cleanly in almost a single direction.
Point a flashlight at a mirror and the light does not scatter; it comes back strongly at one particular angle. What reaches the camera is only "something else reflected by the mirror," with almost no light that conveys the condition of the surface. With rough paper or a matte painted surface, light scatters in every direction (diffuse reflection), so the camera can capture the surface relief. A mirror is the exact opposite. That is why it ends up either "pitch black" or "full of reflections," and the surface cannot be seen.
Polished metal, glass, semiconductor wafers, lenses, plated parts: every material that causes the familiar on-site complaint "it reflects, so we can't see the scratches" shares the same difficulty as this mirror.
How TR-300 works: changing the direction of light to read "surface slope"
At the core of TR-300 is a method called Photometric Stereo. It combines images captured with light from multiple directions, calculates the fine slope (normal) of the surface, and reconstructs the shape of the surface in three dimensions.
Think of how the shadow of a slight step on a dark floor changes depending on whether you shine a flashlight from the left or the right. By capturing many images while changing the direction of the light, the slope of the surface, "this point faces slightly this way," is worked out point by point. The strength of this approach is that it relies not on color or brightness itself but on how the surface is tilted relative to the light.
Now let us look at the results of observing the mirror in three stages.
Stage 1: In 2D (color and brightness), nothing can be seen
First, a 2D image of the mirror captured normally.
2D captured image of the mirror. Due to reflection it is almost pitch black, and the condition of the surface cannot be made out
As you can see, it is almost pitch black. Only the outer edge glows faintly; the inside is solid black. Scratches, waviness, flatness: none of it can be read.
This shows plainly that when you try to look at a mirror on the scale of "color and brightness," the information drops to zero. The subject is not at fault; looking at a mirror through color is itself the problem. This single point is exactly why conventional 2D inspection stumbles on mirror surfaces.
Stage 2: In 3D, "macro flatness" becomes visible
Here is the same mirror converted to 3D with TR-300's Photometric Stereo and shown as a height map.
3D height map of the mirror reconstructed with TR-300. The uniform gray indicates macro flatness
In stark contrast to the pitch-black 2D image, the entire surface is shown in a uniform gray. It may look flat and even dull, but from an inspection standpoint this is a very good result.
In a height map, high and low areas appear as differences in brightness. So an entirely uniform gray means almost no difference in height, which means the surface is extremely flat at the macro level. It is quantitative evidence of that. "It looks featureless because it really is flat." The flatness of a surface that had been hidden by reflection, with nothing to see, becomes visible here for the first time.
Stage 3: With enhancement, scratches invisible to the naked eye emerge
But this is where it gets really interesting. Here is the same 3D height map with contrast and differential enhancement applied, stretching out the tiniest differences in height.
Mirror surface after 3D enhancement. Fine undulations and scratches invisible to the naked eye emerge
Across the formerly uniform gray surface, fine patterns suddenly emerge. Three things are worth noting.
- Orange-peel-like fine undulation: fine waviness like the skin of an orange spreads across the entire surface. In the automotive paint industry this is also called "orange peel," a surface texture almost impossible to notice with the naked eye.
- Vertical scratches (line scratches): thin line scratches can be seen running from around the center toward the right. These are scratches that could never be captured through reflected light.
- Wipe-mark-like unevenness: gentle unevenness, like that left after wiping with a cloth, can also be read.
All of these are extremely fine differences in height on the order of a few µm. Dig further into the surface that Stage 2 showed to be "flat," down to the µm level, and you find that it actually carried undulations and scratches invisible to the naked eye. This reversal from "nothing" to "something" is the power that Photometric Stereo holds for mirror surfaces.
How does this help on the inspection floor?
The capability shown here with a mirror applies directly to the inspection of highly reflective and glossy materials.
- Materials whose scratches were hidden by reflection can be inspected as surfaces: polished metal, glass, semiconductor wafers, lenses, plated parts, and other surfaces previously blocked by reflection can be captured as shape.
- "Looks flat" can be quantified: a uniform gray height map is the starting point for evaluating flatness and waviness numerically. "Roughly flat" can become "how flat."
- Fine defects invisible to the naked eye can be brought to light: fine scratches and wipe marks on the µm order can be made visible to both people and machines through enhanced display.
Recapturing as surface shape the subjects long given up on because "they reflect, so they can't be inspected": TR-300 is a system developed to tackle that difficult ground head-on.
Summary
- A mirror is the hardest subject in visual inspection. A specular surface bounces light straight back, so in a 2D image it is either pitch black or full of reflections, and the surface itself cannot be seen.
- TR-300's Photometric Stereo can reconstruct "the shape of the surface" even on a mirror surface that returns nothing but reflections. The surface emerged in three stages: solid black in 2D, macro flatness in 3D, and, with enhancement, fine undulations and scratches invisible to the naked eye.
- This capability applies directly to scratch detection and flatness evaluation of highly reflective and glossy materials such as polished metal, glass, wafers, lenses, and plating.
If you would like a demo or sample evaluation of surface observation, scratch detection, or flatness evaluation of mirror and glossy materials with TR-300, please feel free to contact us.
