Get Started with TinyML

Very interesting question. For devices that'll never be updated in the field you'll have a similar approach as you do today with rigid testing before deploying, you'll just have more finegrained thresholds and conclusions which your ML model can determine. For devices that have some binary delivery mechanism there are many more possibilities. One interesting thing about TensorFlow Lite is that it can decouple the runtime from the model definition, so you can update the neural network without recompilation, making it easy to iterate on new models. But regardless, this is embedded, so we should always be careful about updates.

In order: 1) walk through the tutorials on vibration and audio analysis (https://docs.edgeimpulse.com/docs/continuous-motion-recognition), 2) For a deeper understanding, read the TinyML Book (https://tinymlbook.com) - written by Dan Situnayake and Pete Warden, 3) If this still interests you I'd consider diving deeper into ML itself. E.g. with https://www.coursera.org/learn/machine-learning (free course by Andrew Ng).

Thanks Miro!

Christopher Seidl explains CMSIS in his "How to get started with Arm Cortex-M55 software development" session this AM.
https://www.embeddedonlineconference.com/session/How_to_get_started_with_Arm_Cortex-M55_software_development

Hi Drew, CMSIS is a vendor-independent hardware abstraction layer for microcontrollers developed by Arm. CMSIS-NN is the library for doing things that many neural networks depend on, like 8-bit matrix multiplication, efficiently in hardware. This will work on any Cortex-M MCU, but naturally processing speed depends on the underlying MCU. It's probably possible to do vibration analysis (like the gesture demo) on an M0 - that demo takes ~4K of RAM at the moment (but I don't have any numbers for inference speed on smaller MCUs), but larger models like audio / still image processing will require a Cortex-M4 and up.

Hi Markus, you can use the Edge Impulse API (https://docs.edgeimpulse.com/reference-link/edge-impulse-api) to automatically retrain your project, and then run all samples in the test set against the new model. This is what some of our users use in their CI pipeline. New data comes in, automatically retrains, and if performance drops they'll get an alert.

Training data depends on your usecase. E.g. keyword spotting requires a lot more data, but simple audio models (scene recognition) can be done in about 20 minutes (e.g. https://www.edgeimpulse.com/blog/audio-based-shower-timer-with-a-phone-machine-learning-and-webassembly/). More is always better though! For one of our customers we have thousands of hours worth of data.

Free for individual developers, SaaS model for enterprises. The latter gives you access to the dataset builder tools, to team collaboration, and allowing you to store datasets on your own servers. All code that is generated will always be royalty-free, no BOM stacking regardless of whether you're using the free or enterprise plan, and no per-device fee (enterprise plan is billed per project/month).

Hi Justin, thanks a lot! No, not my real grandma ;-)

By the way, do you have any interesting industry success stories, any stellar feedback from companies who got so much more from their $40?

The larger players are doing very similar things as we are applying here, mostly around voice (Amazon, Google, Apple) and they see lots of value. But the movement outside of these players has just started up (TensorFlow Lite for Microcontrollers is barely a year old, and CMSIS-NN has been usable for maybe a month or 3) so I think we're still a bit off of large scale success for ML there. We are doing some very promising deployments with customers, but before they have reached scale we'll be some time out.

Also, about the EI (Edge Impulse) dev GUI, I haven't tried it yet but is it possible to export the entire learning algo. via Python? It sometimes needs a little tweeking depending on the nature of the time-series data, adjustments on parameters in PCA and LDA (something of the sort..) Would be awesome to get access to the raw code (we could just run the modified one through our local Python IDE)

Yeah, absolutely. All our signal processing code is here (https://github.com/edgeimpulse/processing-blocks) in Python, and here in C++ (https://github.com/edgeimpulse/inferencing-sdk-cpp). Neural network code is in Keras (just click on the little dots, and select Edit as iPython notebook to get something to run locally). You can always get data from intermediate steps (e.g. after DSP but before Neural Network) from the Dashboard page in .npy files.

Speaking of contributing to the pool of knowledge in ML, does EI need to adhere to any existing standards? Given there are so many ML implementation out there, I'm not familiar if there are any..

Yeah, on the ML side we try to use as many standard things as possible. We use Keras & TensorFlow for training, TensorFlow Lite for inferencing (we're a contributor, and our lead TinyML engineer used to be in the TF team at Google), CMSIS-NN and CMSIS-DSP for hardware optimization, etc. Trying not to reinvent the wheel here, we're a small team :-)

I'm really glad EI removed all the Math involved in ML training to make things more comprehensive (and not just for data scientists). Thanks so much for your wonderful talk!

Awesome, thanks!

Hi Yunus, difficult question. Some customers are doing things with much higher frequency data (e.g. vibration analysis at 1000Hz) and then smaller samples are enough to accurately look at discrepancies (if you have something that rotates at 400Hz you can get an idea that way). I'd suggest to grab some data together and try it out (it's very little data after all!).

Thanks! Yeah, it's a supernifty little board, and we're going to be doing a lot more with it in the near future!

For the feature extraction we use FFT + Power Spectral Density based on Welch's method. See the code here: https://github.com/edgeimpulse/processing-blocks, then paired with a small fully connected neural network (2 hidden layers of 20 and 10 neurons) trained with TensorFlow.

Thank you for this very interesting talk on TinyML and Edge Impulse! Now if you'll excuse me, I'll take a dive into that Webinar on TinyML in your blog :-)

Thanks!