Geolocated Video Analysis and Inter-Object Distance Measurement

Thu, 01 Jan 2026 00:00:00 +0000

Context

Industrial sites are often covered by fixed cameras whose footage is only reviewed after an incident and yields no usable data. The aim was to turn existing cameras, and occasional drone footage, into a measuring instrument: for every detected object, a latitude/longitude position and metric distances rather than a pixel box.

This is hard for three reasons. A single camera gives no depth, so scene scale must be inferred. The usual georeferencing method, a ground-plane homography, assumes a flat floor and fails silently for raised objects or uneven terrain. And high-resolution video is expensive, while the common shortcut of downscaling to 640 px discards the geometric detail ground positions depend on.

Approach

flowchart TB
 A["Video source
file · RTSP · drone clip (up to 3840×2160)"] --> B["Object detector at native resolution
detect · segment · pose · OBB"]
 A --> C["Metric monocular depth model
depth map"]
 B --> D["Multi-object tracker
motion model + association"]
 B --> E["Per-object depth
mask pixels, outlier rejection"]
 C --> E
 E --> F["Ray casting
pixel ray + metric depth + camera pose"]
 F --> G["WGS84 position
optional DEM for ground elevation"]
 E --> H["3D distances
between masks · along trajectories"]
 D --> H
 G --> I["SQLite store
detections · tracks · GPS trajectories"]
 H --> I
 I --> J["Analytics
zones · flows · orthophoto overlay"]

Methods

Detection. A YOLO-family detector, at several model sizes, in four modes (detect, segment, pose, oriented boxes). Inference runs at native resolution with no forced imgsz. Segmentation is the default because masks give cleaner depth samples than boxes. PyTorch, ONNX, TensorRT and CoreML weights are accepted (CUDA and Apple MPS).
Monocular depth. A metric monocular depth model predicts a depth map, which is resampled to the frame size. Each object’s depth is a robust estimate over its mask pixels, after outlier rejection. A validation step compares predicted depth with reference zones whose distance is known, to detect calibration drift.
Geolocation by ray casting. A ray is cast from the camera centre through the object’s pixel and stopped at the object’s metric depth. The resulting point is converted from the camera frame to a local North-East-Down frame, then to WGS84. Unlike a homography, this does not assume a flat ground. An optional DEM supplies ground elevation. Positions can be overlaid on a georeferenced orthophoto for visual checks.
Tracking. A custom multi-object tracker with a motion model, confidence-aware association and a track-fusion step that links fragments using gap, heading and speed limits. When depth is available, each track stores a 3D trajectory, from which the distance travelled and the speed in m/s are derived.
Inter-object distances. Later consolidated into a reusable vision library. Two segmented objects are back-projected into 3D, and the minimum point-to-point distance is reported. Overlapping masks are treated as contact. A Monte Carlo simulation perturbs depth, focal length and pixel positions to give a standard deviation and a 95 % interval. Zone membership uses the share of the mask inside the zone, with hysteresis, rather than a centre-point test.
Engineering. The configuration supports a two-GPU split (detection on one device, depth estimation on the other), RTSP discovery and recording, and processing of several cameras in parallel.

Results

Measured on a single NVIDIA RTX 5090. The detection benchmark covers detection and tracking only, with depth disabled.

Setting	Video	Throughput	Output
Largest detector, batch 1	3840×2160, 50 fps, 3,301 frames	5.1 fps	320,217 detections, 1,941 tracks
Smallest detector, batch 1	3840×2160, 50 fps	13.0 fps	158,989 detections, 908 tracks
Largest detector, batch 1	2304×1296, 15 fps, 5,000 frames	17.9 fps	73,502 detections, 2,706 tracks
Smallest detector, batch 1	2304×1296, 15 fps	39.0 fps	51,890 detections, 1,499 tracks
Full chain (detection + depth + tracking + storage)	2304×1296, 300 frames	2.02 fps	12.3 detections per frame

In the full-chain run, depth estimation took 117.6 s of the 148.3 s total (about 80 %), and detection took 20.9 s. Depth estimation, not the detector, is the bottleneck. On 4K footage the largest detector runs about ten times slower than real time. That suits the intended use, which is analysis of recorded archives, not live alerting.

Limitations & next steps

No geolocation accuracy has been measured against surveyed ground truth yet. The ray-casting stage was added after the timing runs above, so its cost is not included in them.
The metric scale depends on the depth model and on the camera pose and intrinsics. Errors in tilt or focal length propagate directly into ground positions.
The depth model predicts at its own internal resolution, so fine depth detail is limited even when detection runs in 4K.
Next steps: an accuracy study with GNSS-surveyed targets, profiling the two-GPU split, and measuring how accurate Monte Carlo distance intervals are on objects at known separations.

Stack

Python · PyTorch · OpenCV · NumPy · SciPy · pyproj · rasterio · SQLite · CUDA · Apple MPS

Sub-Millimetre Crack Detection in Civil Structures

Fri, 01 Mar 2024 00:00:00 +0000

Context

Non-destructive structural health monitoring for bridges, buildings and infrastructure. The challenge is not only to detect cracks, but to classify them by width — without contact or scaffolding.

Width is what matters for inspection: depending on how open a crack is, the repair (filling) method is not the same. An inspection system therefore has to be very precise, because a wrong width class means a wrong repair specification.

Four width classes

Class	Crack width	Consequence
1	< 0.3 mm	Finest cracks — often narrower than a pixel; dedicated repair approach
2	0.3 – 0.7 mm	Different repair approach
3	0.7 – 1 mm	Different repair approach
4	≥ 1 mm	Different repair approach

Each class calls for its own filling/repair approach, so the classification directly drives the maintenance decision.

The sub-pixel challenge

The key difficulty is to go below the pixel size. A 0.3 mm crack does not even cover a full pixel at typical inspection resolutions — and the system must still detect it, even when the ground sampling distance is somewhat coarser than 0.3 mm.

Why is this possible at all? A crack thinner than a pixel does not vanish: it still occupies part of the pixels it crosses, and slightly alters their intensity compared with the surrounding surface. Taken individually, that partial-pixel signal is weak; but a crack is a continuous, elongated feature, so the same faint contrast repeats coherently along many adjacent pixels. Exploiting that contrast and continuity is what allows detection — and width estimation — beyond the nominal resolution of the image.

Acquisition: drone and smartphone

Drone photography — high-resolution UAV capture of structural surfaces, for large or hard-to-reach structures.
Smartphone photography — the approach also works on images taken with a mobile phone, which I have tested, making close-range inspection possible with everyday equipment.

Pipeline

flowchart TB
 A["Image capture — drone or smartphone"] --> B["Image tiling & calibration"]
 B --> C["Crack segmentation — CV / DL"]
 C --> D["Sub-pixel width estimation"]
 D --> E["Width classification — 4 classes"]
 E --> F["Defect map & inspection report"]

Methods

High-resolution capture of structural surfaces by drone or smartphone.
Computer-vision / deep-learning crack segmentation on concrete and masonry.
Metric calibration and sub-pixel analysis for crack width estimation.
Classification into four width classes matched to repair methods.

Results

Non-destructive, repeatable inspection of civil structures.
Geolocated defect maps supporting maintenance prioritisation.

Models and visuals will be added.

Stack

Python · OpenCV · deep learning · high-res RGB drone · smartphone imagery · photogrammetry

Computer Vision on Boris Dupoux

Geolocated Video Analysis and Inter-Object Distance Measurement

Context

Approach

Methods

Results

Limitations & next steps

Stack

Sub-Millimetre Crack Detection in Civil Structures

Context

Four width classes

The sub-pixel challenge

Acquisition: drone and smartphone

Pipeline

Methods

Results

Stack