Real-Time Gesture Recognition for Resource Constrained Devices using TinyML with TensorFlow Lite

Advances in low power machine learning have enabled intelligent processing directly on microcontroller devices. This talk will present a fully on-device gesture recognition system deployed on an arm-based Arduino Nano 33 BLE board using tinyML with tensorflow lite for microcontrollers. The system demonstrates how accurate motion-classification models can operate within the severe memory and computational limits of an embedded platform without relying on cloud connectivity.

Data is collected using the multi-axis accelerometer sensor on the Arduino nano for various gestures which is then trained separately to build a tensorflow model based on a convolutional neural network (CNN). It is then converted to a tensorflow lite model and deployed on the Arduino nano for gesture recognition, achieving real-time inference at low latency and with minimal power consumption.

Audience will learn practical guidelines to build and deploy machine learning models for small, resource constrained embedded devices. This approach demonstrates the potential of tinyML for enabling intuitive human machine interfaces across wearables, consumer electronics, and industrial IoT sensing applications.

What this presentation is about and why it matters

How do you build gesture recognition that runs locally on a small board, when memory, compute, and power are all tight? Neethu Elizabeth Simon walks through a practical TinyML project built around an Arduino Nano 33 BLE, using accelerometer and gyroscope data to classify hand motions in real time. The talk follows the full pipeline, from data collection and CSVs to training a compact CNN, converting it with TensorFlow Lite, and deploying it on device. It is a grounded walkthrough of the hardware, software, and deployment flow, useful for anyone trying to move ML from notebook demos onto constrained embedded targets.

Who will benefit the most from this presentation

• Embedded engineers prototyping ML features on microcontrollers with limited RAM or no FPU
• Firmware developers who need a concrete example of sensor collection, model conversion, and on-device inference
• ML practitioners curious about the constraints and workflow of TinyML deployment
• ARM or Arduino users evaluating a gesture-recognition project as a starting point
• Teams building low-power, always-on interfaces for disconnected or privacy-sensitive devices

What you need to know

Some familiarity with embedded development and basic machine learning will help, especially around these topics:

• Microcontroller workflows, including flashing firmware and using serial output
• Basic sensor concepts such as accelerometers, gyroscopes, and IMU data
• General ML training and inference vocabulary
• Interest in model conversion and quantization for constrained targets

Glossary (terms used in this talk)

• RTOS (Real-Time Operating System): An operating system designed to provide predictable timing behavior for real-time applications.
• Quantization: The process of mapping a high-precision or continuous value to a limited set of representable values. In digital signal processing, quantization introduces approximation error and can affect accuracy and stability.
• FFT (Fast Fourier Transform): An efficient algorithm for computing the discrete Fourier transform.
• Accelerometer: A sensor that measures acceleration, commonly used in vibration acquisition.
• TFLite (TensorFlow Lite): A lightweight runtime for deploying TensorFlow models on resource-constrained devices.
• Fixed-point: A binary numeric representation with a fixed radix point; the available precision and range are determined by the number of integer and fractional bits.
• Pruning: A model compression technique that removes unnecessary weights, channels, or connections to reduce model size and compute.
• CSV (Comma-Separated Values): A plain-text tabular data format where each row holds a record and columns are separated by commas. It is widely used for exchanging logs and sensor datasets between tools.

Toolbox (mentioned in this talk)

• GitHub: A web-based platform for hosting Git repositories and collaborating on software development.
• Arduino Nano 33 BLE: A small development board featuring Nordic Semiconductor's nRF52840 MCU and built-in Bluetooth Low Energy support.
• Google Colab: A cloud-based notebook environment for running Python code in a browser.
• TensorFlow: An open-source machine learning framework for building and training neural networks and other numerical models.
• Keras: A high-level deep-learning API used to build and train neural networks, commonly associated with TensorFlow.
• ARM Cortex-M: A family of 32-bit ARM microcontroller cores widely used in embedded systems for real-time control, low-power applications, and general-purpose embedded computing.

Final thoughts

Practical and demo-driven, this session gives you a clear sense of what TinyML deployment looks like when it leaves the slide deck and lands on a real board. The value here is a reusable mental model for the end-to-end workflow, plus a better feel for where projects tend to get constrained by data quality, runtime size, and tooling. It will help embedded developers, edge-AI builders, and curious ML practitioners alike. The talk keeps the focus on making intelligence fit the device, not the other way around.

This overview is AI-generated from the session transcript. Spot an issue? Let us know.