Non-destructive testing from a stationary platform is hard. Non-destructive testing from a moving robot platform is harder, and the reasons why reveal something fundamental about why sensor fusion in industrial NDT is not an optimization choice but a necessity.
When a human inspector positions a phased-array ultrasonic probe at a specific location on a pipe wall, they can hold it steady, apply controlled coupling pressure, and acquire a clean signal over multiple seconds. When a robot traverses the same surface at crawl speed, every one of those factors introduces noise: probe position changes, coupling pressure varies, and the acquisition window is measured in milliseconds rather than seconds. Each sensor modality we use on a robot platform operates under versions of this same constraint, and no single sensor handles all the failure modes our system is designed to detect.
This post describes how we think about combining visual, thermal, and ultrasonic inputs and what each modality contributes to the final defect classification.
What Each Modality Sees
Before getting into fusion, it helps to be precise about what each sensor can and cannot detect on its own.
High-resolution visual (RGB camera) is the primary modality for surface and near-surface defects. It detects surface pitting, coating delamination, oxidation banding, visible crack initiation, and mechanical damage. It is fundamentally a surface measurement: it cannot see below the coating layer or below the material surface. What the camera sees is also strongly dependent on lighting geometry. Shallow-angle raking illumination reveals surface texture discontinuities that frontal illumination washes out. A moving robot with fixed lighting faces varying effective illumination angles as it traverses surface geometry, which creates condition-dependent performance variation that a static imaging setup avoids.
Thermal imaging detects temperature differentials across the inspected surface. The useful application in infrastructure NDT is not measuring absolute temperature but identifying localized regions where heat conduction is different from the surrounding material. Active thermography, where the surface is briefly excited with a heat source and the decay pattern is imaged, is the most information-rich approach but requires controlled heating application that is difficult on a moving platform. Passive thermography detects differential heat absorption and retention from ambient thermal cycling. On a pipeline with active insulation, passive thermal imaging is the primary tool for detecting corrosion under insulation (CUI): the insulation's thermal behavior changes where there is moisture intrusion and active corrosion underneath, producing a detectable temperature signature at the insulation surface.
Ultrasonic measurement is the only modality in our payload that provides direct wall thickness data. Pulse-echo ultrasonic testing transmits a sound pulse into the material and measures the time for the reflection from the far wall to return. The wall thickness is calculated from the time-of-flight and the material sound velocity. Thinning from internal corrosion, CUI, or erosion shows up as a reduction in the measured wall thickness below the nominal specification. Ultrasonic also detects laminar defects and certain subsurface crack configurations that are invisible to both visual and thermal inspection.
Why Single-Sensor NDT on a Moving Platform Fails
Consider a scenario: the robot detects an anomaly in the thermal channel, a localized region 15cm in diameter that shows slightly elevated temperature differential compared to surrounding pipe surface. The thermal signal alone gives you the location and approximate size of the anomaly, but not its cause. It could be active CUI with moisture ingress and ongoing oxidation. It could be a void in the insulation with different thermal properties than intact insulation. It could be a region of thicker insulation wrap that retains heat longer. It could be an anomaly in the pipe contents (a partial blockage or flow restriction) affecting local wall temperature from the inside. Those four causes have very different maintenance implications, and thermal alone cannot distinguish them.
Now add the ultrasonic channel at the same location. If wall thickness reads within specification, the CUI hypothesis weakens significantly; you would expect wall thinning if active corrosion were present. If wall thickness is reduced by 10 to 15%, the CUI hypothesis is strongly supported and the finding warrants immediate follow-up.
Add the visual channel. If the insulation surface at the anomaly location shows visible moisture staining, discoloration consistent with prior wetting, or coating degradation, that adds supporting evidence for the CUI hypothesis. If the surface looks intact, the void-in-insulation or flow-restriction hypotheses become more plausible.
The three channels together give a classification confidence that no single channel can approach. This is the core argument for sensor fusion in industrial NDT: it is not primarily about redundancy, it is about the fundamentally different physical phenomena that each sensor accesses.
The Fusion Architecture Problem
Combining data from three sensors with different spatial resolutions, different noise characteristics, and different temporal acquisition rates is not trivial. The outputs need to be registered to a common spatial frame before any fusion can happen.
Visual data comes from a camera with a specific field of view and depth relationship to the inspection surface. Thermal data comes from an infrared sensor with a different (typically lower) spatial resolution and a different field of view. Ultrasonic data comes from a contact probe at a specific point contact, not a 2D image at all. Registering these three to a common spatial reference requires knowing the physical geometry of the sensor array, the robot's position and orientation at each moment of data acquisition, and the mapping between sensor coordinate frames and world coordinate frames.
The position and orientation tracking problem is therefore coupled to the sensor fusion problem. If the robot's pose estimate is off by 50mm at a given moment, the thermal anomaly detected at frame T and the ultrasonic reading taken 3 seconds later at a nearby location are being associated incorrectly in the spatial map. Position tracking accuracy sets a floor on the spatial registration accuracy of the fused output.
The fusion itself is handled at two levels. Low-level fusion at the feature extraction stage, where complementary spatial features from visual and thermal channels are combined in a shared representation before classification. And decision-level fusion, where each modality produces an independent preliminary classification and a gating layer combines them with explicit uncertainty weighting, giving less weight to a modality that is currently operating in a condition where it is known to be unreliable.
Handling Sensor Dropout and Degraded Conditions
A real inspection run includes conditions where one or more modalities underperforms. Visual imaging degrades in sections with condensation on the lens, extreme surface corrosion that produces non-specular reflectance, or rapidly changing lighting geometry at complex structural transitions. Thermal imaging loses sensitivity in sections with high ambient temperature variability or direct solar exposure that swamps the passive thermal signal. Ultrasonic contact measurement is unreliable when the robot cannot maintain sufficient contact pressure on rough or curved surfaces.
A fusion architecture that fails or produces garbage output when any single modality degrades is not practically useful for real-world operation. The system needs to degrade gracefully: when the visual channel is degraded, fall back to thermal-plus-ultrasonic classification with appropriate confidence reduction; when ultrasonic contact is unreliable, produce thermal-plus-visual classifications with a flag indicating that wall thickness was not measured for this section.
This graceful degradation behavior requires explicit modeling of sensor reliability as a function of observable conditions. The current approach uses a reliability score per modality, updated continuously based on signal quality metrics like signal-to-noise ratio in the ultrasonic channel, feature density in the visual channel, and spatial variance in the thermal channel. The fusion weights are modulated by these reliability scores in real time during the run.
What Sensor Fusion Does Not Solve
It would be misleading to present sensor fusion as a complete answer to the NDT problem on a mobile platform. There are failure modes that none of the three modalities we described can detect reliably.
Deep subsurface laminar defects, greater than roughly 30% of wall thickness from the inspection surface, are difficult to detect with the single-element ultrasonic configuration we use. Phased-array configurations that image a volume of material rather than a point measurement would improve this, at the cost of increased complexity and probe size. Hydrogen-induced cracking, which occurs at specific microstructural sites rather than at predictable stress concentrations, presents a detection challenge because it may not produce surface signatures visible to the optical channels and may be below the spatial resolution of single-element ultrasonic at the relevant crack sizes.
We are also not trying to replace laboratory NDT methods on a robot platform. High-temperature creep damage, residual stress measurement, and high-resolution fracture mechanics characterization all require specialized techniques that are not yet practical on a mobile inspection robot at reasonable size and cost. The sensor suite we carry is calibrated to detect the failure modes that are most prevalent in aged transmission steel and pipeline infrastructure, and to generate findings that direct human follow-up to the right locations for more detailed characterization where needed. That is a different and more limited goal than comprehensive NDT, and being clear about that distinction matters for how the system's outputs are used in maintenance decisions.
