AnomalyDINO: A Novel Few-Shot Anomaly Detection Method Built on DINOv2

AnomalyDINO: A Novel Few-Shot Anomaly Detection Method Built on DINOv2

Anomaly detection technology plays an important role in many fields, such as quality control of industrial products and medical image analysis. In particular, when only a small amount of normal data is available (few-shot), conventional anomaly detection methods cannot learn sufficiently, and accuracy drops.

This article introduces AnomalyDINO, an anomaly detection method that uses the pre-trained visual features of DINOv2 and can be applied immediately without training (training-free). AnomalyDINO analyzes images patch by patch and identifies the parts that differ from normal data, achieving high-accuracy anomaly detection with only a few normal samples.

https://arxiv.org/abs/2405.14529v2

What is AnomalyDINO?

AnomalyDINO is a method that uses the visual features of DINOv2 to achieve high anomaly detection performance even in few-shot settings. Its key feature is that, unlike conventional methods, it does not need a large amount of normal data and can be applied immediately with only few-shot data.

Key features

• Training-free – can be applied without additional data or fine-tuning
• Supports both image-level and pixel-level anomaly detection
• Achieves a 1-shot AUROC of 96.6% on the MVTec-AD dataset
• Fast inference (about 60 ms per image)
• Simple implementation that makes industrial use easy

Comparison with conventional methods

Conventional few-shot anomaly detection methods had the following issues.

• Training required – additional data or fine-tuning is essential
• High computational load – large networks must be trained
• Hard to localize anomalies – only image-level classification, making it difficult to pinpoint where an anomaly is

AnomalyDINO addresses these problems by using DINOv2's pre-trained features and computing similarity patch by patch.

Patch-level anomaly detection

The core of AnomalyDINO is detecting anomalies based on the similarity between patches. Unlike conventional methods that learn features of the whole image, it splits the image into small patches and uses the features of each patch, achieving anomaly detection that is both high-accuracy and training-free.

Processing flow

1. Extract patch features of the normal data with DINOv2
   • Split the image into small patches (AnomalyDINO uses 14 × 14 pixel patches) and extract the features of each patch with DINOv2.
   • The extracted features are high-dimensional (for example, 768 dimensions) and retain detailed structural information about the image.
2. Store the normal patch features in a memory bank
   • Store the patch features obtained from the few-shot normal images in a database (memory bank).
   • The memory bank serves as reference data during anomaly detection and is used to calculate anomaly scores.
3. At detection time, compare the test image's patches with the memory bank
   • Extract the features of each patch in the test image with DINOv2 and compare them with the normal patches in the memory bank.
   • Use the k-nearest neighbors method (k-NN) to calculate similarity to the normal patches, and judge patches that deviate greatly from the normal data as anomalous.
4. Calculate the anomaly score
   • Calculate the anomaly score of each patch using cosine similarity.
   • The anomaly score of the whole image is aggregated from the maximum or mean of the anomalous patch scores.

With this approach, AnomalyDINO achieves high-accuracy anomaly detection even in few-shot settings and can localize anomalous regions at the pixel level.

Experimental results

AnomalyDINO was evaluated on the following benchmark datasets.

MVTec-AD (anomaly detection for industrial products)

• 1-shot AUROC: 96.6% (the best previous method: 93.1%)
• Also recorded the highest accuracy in the 2, 4, 8 and 16-shot settings

VisA (anomaly detection for diverse industrial images)

• Outperforms existing methods in the 8-shot and 16-shot settings

  

Inference speed

• An average of 60 ms per image at 448 × 448 resolution (real-time processing is possible)

AnomalyDINO matched or exceeded the accuracy of all conventional few-shot methods while delivering fast inference.

Benefits for industrial use

AnomalyDINO is easy to deploy in industrial and medical applications for the following reasons.

• No training is needed, so it is quick to introduce
• Real-time processing is possible (60 ms per image)
• Simple implementation makes operation and debugging easy
• High-accuracy anomaly detection is possible even with few samples

Possible application areas include the following.

• Semiconductor manufacturing: detecting fine defects to improve manufacturing quality
• Food inspection: quickly detecting foreign matter and quality anomalies
• Medical image analysis: identifying abnormal tissue in MRI and X-ray images

In particular, the fact that it can be applied immediately without training is a major advantage that makes on-site introduction smooth.

Challenges and outlook

Although AnomalyDINO achieves high-accuracy few-shot anomaly detection, the following challenges have also been pointed out.

• Difficulty detecting semantic anomalies
  • It is strong on shape and texture anomalies, but struggles with "semantic anomalies".
• Accuracy may drop when the normal data lacks diversity
  • If too little normal data is registered in the memory bank, false detections increase.

Going forward, new approaches are expected, such as anomaly detection that uses multimodal information (image + text).

Summary

AnomalyDINO is a new approach to few-shot anomaly detection built on DINOv2. In particular, the fact that it can be applied immediately without training is a major advantage for industrial use.

As its use spreads in industrial and medical fields, it may become a new standard in anomaly detection technology.