Written By George Alexakis, FORTH-ICS
Certainly. But advanced imaging technologies must jump over several hurdles.
Robotic waste sorting systems that utilise both RGB and hyperspectral cameras face significant challenges, particularly when dealing with overlapping objects on a moving conveyor belt.
Overlap between waste items not only complicates the visual and spectral analysis but also introduces practical risks to the mechanical sorting process.
When waste items overlap, especially if one object is partially or fully covered by another, both the RGB and hyperspectral cameras struggle to acquire complete and accurate data.
In the images below, the operator transitions from RGB to overlapped hyperspectral images.
RGB cameras may fail to detect object boundaries, while hyperspectral sensors may not capture a full spectral signature—leading to uncertainty in material classification. Since hyperspectral imaging relies on identifying materials based on their unique spectral fingerprints, missing data due to occlusion significantly reduces confidence in the predictions.
Furthermore, items that are underneath an overlap are essentially invisible to both sensors, making it impossible to reliably identify them.
From an operational standpoint, these systems are typically designed to avoid interacting with overlapped objects, especially those that are underneath.
The main reason is uncertainty in material identity—sorting an unknown object could result in contamination of waste streams. Additionally, attempting to grasp or move an overlapped item could disturb or displace the top object, leading to mechanical faults or disruptions to the consistent flow of the conveyor.
This is particularly critical in high-speed sorting environments, where even minor disturbances can cause jams or errors downstream.
Image alignment between the RGB and hyperspectral cameras plays a key role in mitigating these issues.
Accurate spatial registration ensures that data from both sensors correspond pixel-to-pixel, which enhances object segmentation and material classification, especially in cluttered scenes.
With proper alignment, the system can better identify which objects are clearly visible and distinguishable from the background or from other waste.
This helps the system make informed decisions about which objects are safe and feasible to sort, and which should be left for later cycles to avoid process disruption. In summary, while overlapping objects pose challenges in visibility, classification, and mechanical handling, robust multimodal alignment significantly improves the system’s ability to make reliable, safe sorting decisions.