R&D
CENTER
KOENJA

3D Vision and Point Clouds: When You Need Height and Pose

A 2D image is not enough for a robot to pick a part or to measure weld bead height. This lesson explains how to clean up, split and align the point cloud from a 3D camera.

Core4 min1 views
Key points
  1. A 2D image tells you what is where. A 3D point cloud tells you how high it is and in what pose.
  2. Point cloud processing runs in this order: region of interest, downsample, noise removal, normals, plane removal, object segmentation, registration.
  3. Template registration gives the object's 3D pose (position + rotation), which is a coordinate the robot can use directly.
  4. In weld inspection, the bead cross-section profile gives leg length, throat thickness and undercut in mm.

A 2D camera sees the world as a flat picture. To pick the top part from a pile in a bin, or to check that a weld bead is convex enough, you need height. A 3D camera measures distance at every pixel and returns the scene as a point cloud, a set of points.

Types of 3D Cameras

fineAi RoboVision supports these 3D cameras.

  • Pattern light (structured light) — Zivid, Photoneo, EpicEye Pixel Pro. A projector casts a stripe pattern. The system computes distance from how the pattern bends on the surface. It suits precise capture of stationary objects.
  • Laser — EpicEye Laser M V3. It stays stable on highly reflective or dark surfaces.

These cameras output an organized point cloud. The points follow the pixel grid of the image, so a 2D image coordinate maps directly to a 3D point. This means an object region found in 2D with YOLO can be carried over to 3D points as is. Clouds are usually saved as PLY files.

Processing Flow

ROI Downsample Denoise Normals Plane removal Segment Register → pose Hundreds of thousands of points → one object's position and rotation
Processing steps Digital Twin Pick&Place runs after a 3D capture.
  1. ROI (region of interest) — Keep only the space you need, such as the workbench or the inside of a bin.
  2. Downsample — Keep one point per voxel (a cell of a regular grid) to reduce the point count. Later steps run several times faster.
  3. Noise removal — Delete points far from their neighbors (points floating in the air).
  4. Normal computation — For each point, fit a small plane to its neighbors to get the surface direction. You use this to choose the grasp direction.
  5. Plane removal — Find and delete the largest plane (floor, workbench). A common method is RANSAC, which fits planes to random samples. The points that remain are the objects.
  6. Object segmentation — Group nearby points to split them into individual objects.
  7. Registration — Fit a pre-registered template model to each object to get its 3D pose (position + rotation).

The registration result is in camera coordinates. The robot needs robot coordinates, and that conversion is Hand-Eye Calibration.

Stitching Views from Several Directions

A large object or a curved surface cannot be seen all at once. RoboVision mounts the camera on the robot arm and captures while moving along a trajectory. It records the camera pose that the robot reports at each capture, then uses those poses to join the captures into one (pose-guided stitching). This is more stable than matching feature points between images, even on smooth surfaces that have no features.

Weld Bead Inspection

A 2D image shows only the shape of a weld line. In 3D you can measure the cross-section profile. RoboVision computes these values from the bead cross-section, in mm.

  • Leg length and throat thickness
  • Bead width and height, and how convex or concave it is
  • Undercut depth

It compares these values with the quantitative criteria of the ISO 5817 quality levels (B/C/D) to assign a level. The report records the OK/NG reason for each defect.

2D or 3D: Which to Use

Question 2D 3D
Is there a scratch or foreign material on the surface? ✓ Suitable Lower resolution is a disadvantage
Are height, flatness and volume within limits? Difficult ✓ Suitable
Can the robot pick overlapping, stacked objects? Difficult ✓ Suitable
Does processing need to be very fast? ✓ Advantage Relatively slow

Real cells use both. Use 2D to find quickly what is where, then look at only that region precisely in 3D.