Home > On-Demand Archives > Talks >
Object Classification Techniques using the OpenMV Cam H7
Lorenzo Rizzello - Watch Now - Duration: 35:17
Machine Learning for embedded systems has recently started to make sense: on-device inference reduces latency, costs and minimizes power consumption compared to cloud-based solutions. Thanks to Google TFLite Micro, and its optimized ARM CMSIS NN kernel, on-device inference now also means microcontrollers such as ARM Cortex-M processors.
In this session, we will examine machine vision examples running on the small and power-efficient OpenMV H7 camera. Attendees will learn what it takes to train models with popular desktop Machine Learning frameworks and deploy them to a microcontroller. We will take a hands-on approach, using the OpenMV camera to run the inference and detect objects placed in front of the camera.
Well, that is definitely an interesting application, I don't know if you could achieve better results using a thermal camera for example, have you thought about it? The OpenMV Cam supports a FLIR thermal camera. One thing I can tell you though is that models running on MCUs at the moment support low-resolution images (talking about ~100x100 to ~200x200) and run at about 5 FPS. Being low-cost though one could think about a network of cameras, but maybe that is hard to maintain or deploy
Hi Lorenzo, great session! Im an embedded sw engineer so I am pretty new to the ML side of things but I wanted to get your thoughts on how to optimize a mobilenet for improved accuracy. I have seen suggestions ranging from making more of the hidden layers re trainable to better image preprocessing. From your experience what do you find is the best place to start when tweaking these parameters? Thanks!
first of all thank you! I am pretty new to the ML side, too, and one thing I want to do in the next months is definitely tweaking parameters and working on different datasets to see how the training process is affected. I will share the results through blog posts or something similar, so stay tuned for updates :)
At the beginning, I wasn't even sure it would be possible to have networks trained through transfer learning working on TF lite micro. Anyway, I think better results can be achieved just by preparing a good dataset: focus on the dataset is something I underestimated at the beginning, but it is, on the contrary, key for a good learning process. Then, you are right, you can work on fine tuning unfreezing hidden layers, but I would start from the dataset.
sorry to everyone commenting, I was on the page, but the comment section was not updating, so I am starting to reply now
Hi, I'll be here to answer questions live from 12PM to 14PM EDT!
I wonder what the success rate would be, using this system, for locating small animals that are mainly trying to hide in an attic (they are making noise, and maybe doing damage). The technical challenges are that cameras work poorly when it is very dark. Turning on lights makes those animals hide, so that's not a solution. And, the animals move around, making it hard to know where to point the camera, which has a limited field of view. So some kind of tracking mechanism would be needed, or it would have to have a 360 degree panoramic lens. I wrote some movement-detection software, and recorded 40,000 images from a 4K camera. But the rodents hide very well and managed to be far from the camera. I thought of machine learning, but the lack of data would make it quite hard to succeed in identifying anything in this case. But over time maybe an affordable system will be able to detect all that in a reliable way, 24hrs a day, and notify the owner as to the type, and quantity of rodents, that are affecting the house.