Optimizing Waste Handling with Interactive AI

Abstract

Automated construction and demolition waste recognition is difficult because material categories appear in cluttered, variable environments. Generic segmentation models do not transfer reliably to this domain. This work develops an interactive pipeline that lets users guide segmentation using bounding boxes, points, or text.

Approach

The system combines foundation-model segmentation with domain-specific prompting. Instead of requiring detailed pixel-level annotation for every object, an operator can draw a box, click a point, or describe the target material. This turns segmentation into a lightweight human–AI interaction.

Results

The approach reaches class-wise performance of roughly 70% for several waste categories and improves on the evaluated state-of-the-art baselines by 9% on average. The different prompt modes make the system useful across varying annotation and operational constraints.

Why it matters

Faster, more accurate material recognition can support sorting and resource recovery in waste-processing facilities. The work also illustrates how interactive foundation models can be adapted to specialised visual environments.