Are You Really "Seeing" This Coin? Separating Color and Shape with FORESIGHT STEREO Viewer
kotai2003The coins we handle almost every day. Their surfaces should be familiar, but are we really "seeing" them? FORESIGHT STEREO Viewer is a GUI application that turns our 3D imaging technology FORESIGHT STEREO® into something people on the shop floor can operate on the spot. We captured a UK £2 coin just once. First, compare the two images.
On the left, what you "see"; on the right, what is actually "captured"
[Left: the LIVE image. The color difference between the gold-colored outer ring and the silver-colored inner disc of the UK £2 coin is clearly visible, but the surface looks almost flat and fine scuff marks are barely discernible. Right: 3D Surface. The color disappears and only the shape remains; the gold/silver boundary vanishes, and countless fine scuff marks appear across the entire surface]
Same coin, same single capture. On the left is the LIVE image, just as the camera captured it. The color difference of the bimetal construction (outer = gold-colored ring / inner = silver-colored disc) is clearly visible, and the inscription ELIZABETH·II·DEI·GRA·REG·FID·DEF is legible. However, the surface looks almost flat, and fine scuff marks are barely discernible.
On the right is 3D Surface (an output that reconstructs only the shape of the surface, that is, its relief), extracted from the same data.
What disappeared, and what appeared
Two things happened in 3D Surface.
What disappeared: the gold/silver boundary. The border between the outer ring and the inner disc, so distinct in the left image, cannot be found on the right. It was "color," not "shape." This is visible evidence that the color information has indeed been removed.
What appeared: scratches and a micro-inscription. Countless hairline scratches (fine scuff marks) emerged across the entire surface, on both the portrait and the field. The lettering, the crown, each strand of hair and the edges of the bead row rise sharply, and dents and deformation on the coin's rim also appear as shapes. And at the lower left of the portrait, the micro-inscription IRB (the designer's initials), which looked like nothing more than a black dot in the LIVE image, can be read as letters.
[Enlarged comparison around the micro-inscription IRB. The inscription, barely discernible in the LIVE image on the left, can be read as letters in 3D Surface on the right]
Above is an enlargement of the same area from the same capture (cropping only). On the left, a small trace; on the right, the letters IRB.
What you "see" and what is "captured" are different. What you think you see with the naked eye and the information that actually exists and can be captured are two different things.
In earlier articles, we introduced dents in steel sheet (defects invisible in color that appear in 3D) and discoloration on semiconductors (color anomalies that disappear in 3D). This time, both are happening at once on a single coin.
This is a single radio button
None of the images so far involve special processing or laboratory procedures.
[The FORESIGHT STEREO Viewer operating screen. On the left, the coin in 3D Surface view; on the right, settings panels for Show Mode / Enhance Mode / Preprocess Mode / Auto Object Alignment / Image Save Mode / Control; at the bottom, previews of three outputs]
This is the FORESIGHT STEREO Viewer screen. Under Show Mode in the right panel are three radio buttons: 2D Texture / 3D Surface / Surface Normal. The "disappeared / appeared" you saw earlier is simply what you get with 3D Surface selected.
The star of this article is not the coin but this screen. The technology's parameters are exposed on screen as things to operate, and people on the shop floor can work with them themselves. That is what this application is about.
Four outputs from one capture
- LIVE image: the appearance as the human eye sees it
- 2D Texture: only the color and pattern of the surface (albedo, meaning the material's own color and reflectance), with shading turned off
- 3D Surface: reconstructs only the shape (relief) of the surface
- Surface Normal: encodes the orientation of the surface (the normal, meaning which way the surface faces) as color
[2D Texture. In grayscale with the shading removed, the boundary between the outer ring and the inner disc remains as a subtle difference in tone]
In 2D Texture, the shading has been removed, leaving only the lightness and darkness of the material itself. It looks even flatter than the LIVE image, and the shape information is almost entirely lost. Between the outer ring and the inner disc, on the other hand, the boundary remains as a subtle difference in tone. It is the color layer, after all. The boundary disappeared only in 3D Surface.
[Surface Normal. The tilt of the surface is encoded as color based on blue-violet to wisteria, and the rising edges of the inscription and beads are outlined in cyan / magenta]
Surface Normal is the source data for 3D reconstruction. The tilt of the surface is encoded as color based on blue-violet to wisteria, and the rising edges of the inscription and bead row are outlined in cyan and magenta. You can see the overall curvature of the coin appear as a smooth gradation toward the rim.
Behind the scenes: eight raw images, one per lighting direction
Check Save Images (Each Channel), and eight raw images, one per lighting direction, are saved as they are. These are the "raw materials" of the outputs, which we have not shown in previous articles.
[A 4×2 grid of the eight raw images, one per lighting direction. Same coin, same position, but the lighting direction differs in each image, changing which side the highlights and shadows fall on the crown, hair and rim lettering]
Same coin, same position, same orientation. The only difference is the direction of the light. You can see the highlights and shadows switch sides on the crown, the hair, the rim lettering and the bead row.
Comparing ch01 and ch05 in particular makes the switch clear.
From these differences in shading, the orientation of the surface (the normal) is solved for each pixel, and from that, shape and color are separated and calculated. This is the principle of Photometric Stereo and the foundation of FORESIGHT STEREO®. Being able to output the eight images also means this foundation can be verified as it is.
An open design
FORESIGHT STEREO Viewer is not a black box.
- You can plug in your own Python preprocessing (
Preprocess Mode) - You can save the eight raw images, one per lighting direction, as they are (
Save Images (Each Channel)) - It has an
AI Masktab (in this article, we only go as far as noting that the tab exists on screen)
Build your own know-how in as preprocessing, or extract the raw data and verify it independently. It is a design open to such uses.
What it means on the shop floor
- Observing shape defects without color contrast: see features that are hard to catch by color, such as dents, depressions and scratches, as shapes
- Observing glossy surfaces: check shading and the material's own lightness and darkness separately
- Comparing limit samples: line up and examine the pass/fail boundary from the common perspective of shape
- Verifying inspection conditions in advance: staff can switch outputs on the spot and try conditions
In every case, rather than "this can detect it," it can be used as a means of observation that supports the eyes of people on the shop floor.
Summary
The images in this article are results of an observation demo on a single UK £2 coin. They do not guarantee performance for all objects and conditions, nor are they measurement results for dimensions or depth. The images shown are visualizations of the object's shape.
With that said, there is one thing we want to convey. The idea of looking at color and shape separately no longer belongs only inside the laboratory. With a single radio button, you can produce it from a PC on the shop floor.
For those who would like to try FORESIGHT STEREO Viewer on their own workpieces (shape defects hard to see by color, glossy surfaces, samples where judging against limit samples is difficult, and so on): TOMOMI RESEARCH, INC. welcomes consultations on demos and sample evaluations.
