Tiny Machine Vision: behind the scenes
Tiny devices, like the ones suitable for low-power IoT applications, are now capable of extracting meaningful data from images of the surrounding environment.
Machine vision algorithms, even Deep Learning powered ones, need only a few hundred kilobytes of ROM and RAM to run. But what are the optimizations involved to execute on constrained hardware? What is it possible to do, and how does it really work?
In this session, we will focus on the capabilities that are available for Cortex-M microcontrollers, starting from the user-friendly environment provided by EdgeImpulse to train and deploy Machine Learning models to the OpenMV Cam H7+.
We will guide attendees through the process using a straightforward example that illuminates inner workings so that attendees can get a grasp on technologies and frameworks. Attendees will walk away understanding the basic principles and be able to apply them not just to the Cortex-M but beyond.
Ciao Andres, thank you and great question! You are right, these sensors are definitely power hungry, and I have to admit I never tried lower power image sensors, is there anything in particular you would suggest? On the other hand, yes, I talk about constrained mostly considering memory, and processing power and I think that for their size, cost and ease of deployment they already represent a big step forward for a lot of applications even if connected to a power line and not battery powered (e.g., ST suggests to use them inside street lamps to turn them on when a person is detected reducing false positives compared to motion sensors). There are also some battery powered applications where bigger batteries are used, for example to power motors, where the consumption is acceptable and the advantage of being able to process on this kind of devices compared to "bigger" ones is certainly noticeable.
p.s. congratulations for your talk, too, I found it incredibly interesting (for everyone looking at this comment, go check it out ;) )
Ciao Lorenzo,
Thanks for the advertising :)
I have worked with some low power image sensors. Most recently, the HB01B0 from HiMax. I have used that in conjunction with RISC-V based processors from GreenWaves, which has enough processing power to run CV tasks with a high framerate at only a few mW. I would say this space of (long-term) battery-operated vision sensors is only now become mature enough for real deployments, which makes it very interesting!
Best,
Andres
Thank You for Talk
Thank you!
Thank you for the informative presentation, I appreciate it!
thank you for attending it!
Ciao Lorenzo,
Nice talk! I have a question about power consumption. You mostly talk about constrained devices in term of memory and processing, but the OV7725 sensor in OpenMV modules is quite power hungry (>100mW). Have you used lower power image sensors? I think energy consumption is a big limitation in vision-based sensors, since they really limit the lifetime of their deployments.