R&D
CENTER
KOENJA

Object Detection: YOLO and RT-DETR Give Defects a Name and a Location

This lesson explains object detection, which answers what is there, where, and how sure the model is, for defect types you already know. It covers box, class and confidence, IoU and NMS, per-class thresholds, and how to use it with anomaly detection.

Intro4 min1 views
Key points
  1. Object detection gives each result a box (location), a class (type) and a confidence (how sure the model is).
  2. IoU is how much two boxes overlap. NMS is post-processing that keeps only the most confident box among overlapping boxes on the same object.
  3. Defects differ in risk, so set the threshold separately for each class.
  4. Use object detection for known defects and anomaly detection for unknown ones. In the field, use both together.

Anomaly detection tells you "this spot differs from normal." To improve a process, though, you need "this is a dent, it is at the upper right, and there were this many today." Sorting and counting defect types is the job of object detection.

Output: Box, Class, Confidence

scratch 0.91 dent 0.84 Box · class · confidence IoU = overlap area ÷ union areaAt 0.5 or more, often treatedas the same object
Left: each detection carries a box, a class and a confidence. Right: the IoU of two boxes is used both to compare with ground truth and to remove duplicates.
  • Box (bounding box) — The position and size of the rectangle around the defect.
  • Class — The defect type. It is a name set at training time (scratch, dent, missing, etc.).
  • Confidence — A value from 0 to 1 for how sure the model is of its call.

IoU and NMS

A model often outputs several slightly different boxes for the same defect. NMS (Non-Maximum Suppression) keeps the box with the highest confidence and deletes the other boxes that overlap it a lot (IoU at or above the limit). The Multi-Inspector modeN_iou setting is this limit. Lowering the value can merge two defects that sit close together into one. Raising it can count the same defect twice.

IoU is also used in evaluation. The IoU of the model's box and the ground-truth box must be at or above a limit (often 0.5) to count as "found correctly."

Per-Class Thresholds

Not all defects carry the same weight. A crack that affects function must be caught on slight suspicion, while a small cosmetic stain may be caught only when the model is sure. Multi-Inspector EB v2.3 uses a different confidence threshold for each class. It also keeps detections below the threshold on screen, with a thin box and a (<thr) mark, so the operator can see borderline cases. All detections are recorded in the YOLO_Detections field of the CSV.

YOLO and RT-DETR

  • YOLO — Looks at the image only once and predicts all boxes at the same time. It is fast and light, so it is widely used for field inference. fineAi uses the Ultralytics YOLO family (v8, v11, etc.).
  • RT-DETR — A real-time detection model based on a transformer. It outputs boxes directly without NMS, which can help with overlapping objects.

fineAi OD Workbench trains 28 models of the YOLO and RT-DETR families on the same data and compares accuracy and speed side by side. Small models are fast and large models are accurate, so pick the most accurate model that fits within the line takt time.

Data and Labeling

Object detection is supervised learning, so labels decide quality.

  • Draw boxes tight to the defect, by one common standard. If each person draws boxes differently, the model wavers too.
  • Write down the class names and the borderline cases (is a small scratch a scratch or ignored?) in a document, then label.
  • Also mix in normal images with no defects. This reduces false rejects (overkill: good parts flagged as defective).
  • Save labels in a standard format such as COCO so you can move them between tools.

When You Need a Shape, Not a Box

A long, curved defect such as a weld bead or a crack leaves a lot of empty space in a box. For these, use a segmentation model, which paints the region pixel by pixel. RoboVision weld inspection finds the defect region by segmentation, then measures quantitative values such as length and depth and judges them by the ISO 5817 level rules.

Using It with Anomaly Detection

Anomaly detection Object detection
Training data Normal only Defects + labels
Defects never seen before ✓ Catches them May miss them
Defect name Not given ✓ Given
Defect statistics and root-cause analysis Difficult ✓ Easy

A natural order is to start a line with anomaly detection, then add object detection by labeling the NG images that pile up during operation. Multi-Inspector runs both modes on one part at the same time.