R&D
CENTER
KOENJA

Finding Anomalies from Normal Images Only: Anomaly Detection and PatchCore

Defect samples are hard to collect, and tomorrow brings a defect you have never seen. This lesson explains anomaly detection, which trains on normal images only and finds what differs from normal, using the widely used PatchCore as the example.

Intro5 min0 views
Key points
  1. Anomaly detection learns only what is normal, not defects. So it also catches new defects unseen in training, because they differ from normal.
  2. The output is a per-pixel anomaly heatmap and one score for the image. If the score is at or above the threshold, the result is NG.
  3. PatchCore stores the features of small pieces (patches) of normal images in memory. It measures anomaly by how far a new patch is from its nearest normal patch.
  4. Two thresholds create three zones: OK, Gray and NG. A person checks ambiguous Gray images, and the result goes back into training.

Three Generations of Inspection

Gen 1 · Rule-based Gen 2 · Supervised learning Gen 3 · Normal-only learning
Training data None (a person designs the rules) Thousands of defect images + labels Tens to hundreds of normal images, no labels
Defects never seen before Pass if not in the rules Weak on types unseen in training Detected as "different from normal"
Setup time Weeks (expert tuning) Weeks to months (collecting defects) Days

On a factory floor the defect rate is usually low. Collecting thousands of defect images takes a long wait, and new types appear while you wait. Anomaly detection turns this problem around. Show the model enough normal examples, and treat everything that departs from normal as an anomaly. Some few-shot methods start with as few as 2 to 30 normal images.

Common Processing Flow

All 11 algorithms in fineAi AD Workbench follow the same frame. Only ③ and ④ differ between algorithms.

① InputRaw image ② PreprocessSquare pad ③ FeaturesCNN · ViT ④ AnomalyDistance · prob. ⑤ MapUpsample·smooth ⑥ Scoremax · top q% ⑦ Decisionscore≥thr→NG Differs per algorithm
② Preprocessing pads the image to a square while keeping its aspect ratio. A long part such as 5:1 is not squashed, so the defect shape is preserved.

PatchCore: A Memory of Normal Pieces

PatchCore has high precision, so AD Workbench recommends it by default. The idea is surprisingly simple.

Training: collect normal patches

  1. Feed the normal images to a neural network pretrained on large-scale images (WideResNet50-2) and take the feature maps of the middle layers (layer2, layer3). One cell of a feature map summarizes one small piece (patch) of the original image as a vector of numbers.
  2. Collect the patch vectors of all patches of all normal images into a memory bank.
  3. A bank that is too large makes search slow, so keep only a subset that represents the whole distribution (coreset sampling).

The network weights are not trained again. The method only scans the normal images once and collects, so training takes a few seconds to a few minutes.

Inspection: distance to the nearest normal

  1. Extract patch vectors from the new image in the same way.
  2. For each patch, find the k nearest normal patches (k=3) in the memory bank and measure the distance.
  3. A patch that looks like normal has a small distance. A patch with a scratch has no similar normal patch, so its distance is large. This distance is the anomaly value at that position.
  4. Scale the per-patch anomaly values up to the original size and smooth them to get the heatmap. The heatmap maximum (or the mean of the top few %), normalized to 0–1, is the image score.
Memory bank (normal patch features) Normal patch → small distance Scratch patch → even nearest normal is far Heatmap · score 0.71 ≥ 0.50 → NG
Anomaly is how different a patch is from the most similar normal patch. Even defects never seen in training get a large distance.

Thresholds and the Gray Zone

Once you have a score, you compare it with a threshold to decide. Lowering the threshold reduces missed defects (escapes, or missed defects) but increases good parts flagged as defective (false rejects, or overkill). Raising it does the opposite. The lesson Model Evaluation and Field Operations covers this balance.

fineAi uses dual thresholds.

  • A score below the low threshold is OK.
  • A score above the high threshold is NG.
  • Between them is Gray, which a person checks.

A person's decision on a Gray image becomes training data again. The more you operate, the fewer Gray images remain.

Where Anomaly Detection Fits Poorly

  • Targets with large variation in normal itself — If the range of normal is wide, as with produce whose shapes all differ, it is hard to tell defects apart.
  • When you need the name and location of a defect — Anomaly detection tells you "this spot is abnormal", but it does not classify "this is a scratch". In that case, use Object Detection alongside it.

The next lesson, Choosing an Anomaly Detection Algorithm, covers how to choose among the 11 algorithms.