What Is Not Captured Cannot Be Detected: Our CTO Lectures at JSME Annual Meeting Workshop W252
kotai2003On Monday, September 7, 2026, at the workshop W252 "Raising Issues and Presenting Case Studies in the Maintenance and Reliability Enhancement of Machinery and Infrastructure" (organiser: Hirotsugu Inoue, Institute of Science Tokyo), part of the 2026 Annual Meeting of The Japan Society of Mechanical Engineers (JSME) held in Building 19 of the Tokai University Shonan Campus, Dr. Seong-Hun Choe, Director and CTO of TOMOMI RESEARCH, Inc., gave an invited lecture.
The title was "Bringing 'AI That Does Not Overlook' to the Shop Floor: Strengthening the Reliability of Maintenance through Image Anomaly Detection". What we wanted to convey comes down to one line: What is not captured in the image cannot be detected. That is why we design the light before the AI.
In this article, we share the content of the lecture itself, why light design comes first, why we train only on good parts, and why we measure without stopping the line, together with the figures and measured values presented on the day.
We have also published the full slide deck from the lecture. It is the same content shown at the venue. Please read it together with this article.
📄 Download the lecture slides (free): https://www.tomomi-research.com/download/jsme2026-ws
Cover of the lecture slides, with the title and event details
Invited under the "AI" theme: W252 as a forum
W252 was a workshop aimed at technical exchange across disciplines, and its programme was organised into three themes.
- Theme 1: Monitoring — Track inspection with Shinkansen service trains (Central Japan Railway Company), self-powered vibration monitoring of bridges (Kansai University), soundness evaluation of concrete deck slabs on road bridges (Toshiba)
- Theme 2: Robotics — A snake-type robot arm for infrastructure inspection (HiBot), application of maintenance robots to nuclear and oil and gas facilities (Mitsubishi Heavy Industries), a peristaltic in-pipe inspection robot (Chuo University)
- Theme 3: AI — Our lecture (16:10 to 16:40), expectations for AI in the inspection industry (Amic), taking on the challenge of AI (Just)
Under the "AI" theme, alongside railways, bridges, nuclear plants and in-pipe robots, we were invited as the only invited lecturer. Why a company working on visual inspection in factories was called to a forum on infrastructure maintenance connects to the "concrete surface" topic in the second half of this article.
Design the light before the AI
When AI visual inspection does not work well, the cause tends to be sought in the model. In practice, however, it often lies one step earlier: in how the image is captured.
Inspection proceeds as lighting, angle, lens and focus/exposure, then image, then AI model, then judgement. Information lost at the very top of this flow cannot be recovered by any downstream step, because a scratch that produces no difference in pixel values cannot be detected by any model.
Under frontal diffuse lighting, the same scratch produces no pixel difference and the AI judges it "good" (a miss). Under low-angle oblique lighting, it stands out as contrast and the same model judges it "defective"
Only the lighting differs between left and right; the model is the same. In other words, without replacing the model, the scratch becomes detectable through the way it is captured alone. Before raising the accuracy of the AI, there is something to check first.
Capture shape, not colour: FORESIGHT STEREO®
That said, choosing a single light cannot cope with scratches in every orientation. That is why we developed FORESIGHT STEREO®. Combining the principle of Photometric Stereo with our own image processing technology, it captures images while illuminating the object from several directions in turn, and from the changes in brightness solves for the "direction the surface faces" at each pixel. The principle fits in one line.
I = ρ ( n · L )
I is the observed brightness (known), L is the direction of the light (known), n is the surface direction we want to find, and ρ is the reflectance. The unknowns are two for n and one for ρ, three in total. Three equations are enough to solve them, so the reasoning is that with three or more lights, the surface direction can be determined.
There are three kinds of output.
- Normal map — the surface direction itself
- Albedo — the pattern alone (without shading)
- Height and curvature — the relief alone (without pattern)
The important point is that scratches and dents appear in "height". Patterns such as colour unevenness, oil stains and printing are separated into the albedo and do not affect the shape channel. Because colour and shape are acquired separately in a single pass on the conveyor, neither process steps nor stations increase.
A 500-yen coin. Fine dents buried in colour in the normal image (left) appear as shape in the 3D Surface (right)
What changed in measurement: the undecidable band shrank by about 80%
We also showed the results of checking the principle with numbers. The comparison covered 128 units of the same part (35 good / 93 defective), with everything identical except the input channel. The model parameters, training conditions and test set were all the same; only the input was switched from the LIVE image (intensity of reflected light) to the 3D Surface (surface shape).
| Metric | Normal lighting (LIVE) | FORESIGHT STEREO |
|---|---|---|
| ROC-AUC | 0.935 | 0.970 |
| Optimal threshold | 0.336 | 0.227 |
| Undecidable band (width of overlap between good and defective scores) | 0.379 (0.340–0.719) | 0.078 (0.183–0.261) |
| Images requiring human review (of 128 in total) | 105 | 42 |
| Automatic NG / automatic OK | 0 / 23 | 75 / 11 |
ROC curves. Normal lighting AUC = 0.935 / FORESIGHT STEREO 0.970
The AUC improvement from 0.935 to 0.970 matters, but what the lecture placed at the centre was the undecidable band. The band where the score distributions of good and defective parts overlap, a region where wherever the threshold is drawn, neither verdict can be stated with certainty, shrank from 0.379 to 0.078, a reduction of about 80%.
Width of the undecidable band: reduced by about 80%, from 0.379 to 0.078
This overlap cannot be closed however the threshold is adjusted. Moving the threshold only increases either misses or false detections; the total amount of ambiguity does not change. The only way to reduce it is to increase the information in the input, and that was the choice of "changing how the image is captured".
In fact, the number of images requiring human review fell from 105 to 42 out of 128, and the same good part that scored 0.37 (a false alarm) under normal lighting scored 0.20 in the 3D Surface and fell into automatic OK.
[Note] In this test set, defective parts account for 73%, which differs from the ratio of good to defective parts on a real line. The figure of 105 to 42 is the breakdown under this condition and does not guarantee the reduction rate on a real line.
Learn only from good parts: E3 ENGINE®
The other pillar is our anomaly detection engine, E3 ENGINE®.
What often happens on the floor is that "images of defective parts cannot be collected". Defects occur only rarely and look different each time, so trying to teach defects through supervised learning hits a wall at this point.
E3 ENGINE therefore learns only the "normal" of good parts and measures deviation from it. No defect training data is needed, and the score is calculated as the nearest-neighbour distance to the normal data. The advantage of this approach is that it can respond even to unknown defects.
The three steps of good-part learning: (1) learn only from good parts, (2) build the normal range, (3) judge and output an anomaly map
In the lecture, this idea was explained with the opening of Tolstoy's Anna Karenina (1878).
Happy families are all alike; every unhappy family is unhappy in its own way.
The happy families are the good parts, and the unhappy families are the defects. Good parts all resemble each other, so they can be learned. Defects differ every time, so they can never all be collected. So we learn from the side that resembles itself: the design philosophy of anomaly detection can be explained exactly by a sentence written nearly 150 years ago.
The output is not just a judgement score; an anomaly heat map shows "where" the anomaly is.
A piston head. Machining marks are distinguished from scratches, and the anomalous area is shown as a heat map
See without stopping: TR-300
The third pillar is about equipment. It started with a scene we saw on a steel plate line we visited. Every time an inspection was needed, the equipment was stopped and a person visually checked only the head of each lot.
TR-300 is a system that analyses the 3D shape of glossy surfaces in real time while they keep flowing, without stopping the line. It implements "moving-object Photometric Stereo", driving four-direction LED lighting and a line scan camera with our own optical synchronisation control.
- Lighting switching cycle: about 100 μs (more than 10,000 times per second)
- Synchronisation accuracy: on the order of microseconds
- 3D shape generation: about 30 ms
- Processing including AI judgement: 60 to 80 ms
- Inspection of a steel plate line while flowing at 40 m/min
TR-300. Moving-object Photometric Stereo with four-direction synchronised LED lighting
Steel plate. Linear flaws and dents that are buried in the pattern and cannot be distinguished stand out as shape
If you cannot measure without stopping, inspection cannot cover every part.
Factory inspection technology works directly for infrastructure inspection
This is also why we spoke at W252.
When a concrete surface is photographed with an ordinary camera, colour and pattern are mixed with the actual relief. Whether something is dirt, a shadow or a crack is hard to separate from the image alone. This is why visual inspection tends to depend on experience.
Captured with FORESIGHT STEREO, colour and shape are separated. In the normal map, cracks, chips and the relief of aggregate appear purely as shape. Because dirt and discolouration do not affect the shape channel, the surface condition can be quantified without relying on the subjectivity of visual checks.
Concrete surface. In the normal image (left), colour, pattern and relief are mixed, but in the normal map (right), cracks, chips and aggregate appear as shape
A technology built to capture factory parts also works, as it is, on the surfaces of infrastructure structures. That was where the lecture landed.
We also presented cases of a 500-yen coin, an automotive emblem, a 6-inch silicon wafer, an EV battery cell, a piston head and a mirror-finish mirror. For mirror surfaces, we added the caveat that a perfect mirror surface is outside the scope in principle.
EV battery cell. Scratches invisible on the white glossy surface appear as linear scratches, unaffected by printing or labels
After the lecture
We also took part as an invited lecturer in the general discussion after the lectures (17:10 to 17:50). A technology poster (A1) and brochures were displayed at the back of the venue.
For those who could not attend on the day, or who could only watch part of it online, we have published the lecture slides.
You can view the same material as on the day, from principle diagrams and application cases to the ROC curves and the measured values of the undecidable band.
📄 Download the lecture slides (free): https://www.tomomi-research.com/download/jsme2026-ws
On the previous day, Sunday, September 6, Representative Director and President Tomomi Sato and CTO Seong-Hun Choe spoke at the conference's special event "Company Day" (Shin-Yokohama Grace Hotel), making it two consecutive days of presentations.
Summary
The lecture closed with the following three lines.
2D → 3D: increase the quantity observed
Classification → good-part learning: learn from the normal side
Stationary objects → moving lines: see without stopping
The three pillars: FORESIGHT STEREO / E3 ENGINE / TR-300
What is not captured in the image cannot be detected. That is why we design the light before the AI.
If you face problems such as "we introduced AI but it still misses defects" or "there are so many false alarms that human review does not go down", start by talking to us about actually capturing your parts. As we said at the venue, bring us the object, and we will capture it together and look at it together.
-
Lecture slides (PDF, free download): https://www.tomomi-research.com/download/jsme2026-ws
- Technology lab: https://www.tomomi-research.com/lab
- TR-300 demo video: https://youtu.be/URBgXEO6MEs
- Contact: info@tomomi-research.com
TOMOMI RESEARCH, Inc. (Fujisawa, Kanagawa, Japan) is "a company that builds a stand-in for the human eye", carrying out research and development in AI image processing as well as optical design and equipment manufacturing in-house.
