Detecting rooftop ice buildup using an Arduino Portenta and synthetic data

In areas that experience plenty of cold weather, icicles and ice dams can present a very real danger to the people and property nearby. In response, Eivind Holt has developed a computer vision-based system that relies on an Arduino Portenta H7, a Portenta Vision Shield, and a slew of AI tools/models to recognize this ice buildup. Best of all, the board’s low power consumption and LoRaWAN connectivity means it can be deployed almost anywhere outdoors.

Before a model can be created, it needs copious amounts of training, data which normally comes from manually-annotated, real images. But recent advancements have allowed for synthetic datasets to be used instead, such as with NVIDIA’s Omniverse Replicator. It was in here that Holt programmatically added a virtual house and randomized icicle models, as well as configured Omniverse to move the camera around a raytraced scene in order to snap virtual pictures and annotate them with the correct label.

Once the realistic, synthetic data had been created, Holt exported everything to Edge Impulse and trained an object detection model for the Portenta H7, although it was also tested in NVIDIA’s Isaac Sim environment via the Edge Impulse extension prior to deployment. Alert generation was achieved by connecting the LoRaWAN radio to The Things Stack and sending a small, binary payload every ten seconds if any icicles were detected.

More information about this project can be found in its Edge Impulse write-up.