Debugging & Testing Embedded Software Through Instrumentation

Instrumentation-based debugging is a highly effective methodology, though it seems underutilized within the embedded systems community. In this presentation, I will introduce Scrutiny, an open-source software suite developed as a personal initiative, designed to streamline debugging, telemetry, and hardware-in-the-loop (HIL) testing for embedded devices.

What this presentation is about and why it matters

How do you debug embedded software when a probe is costly, intrusive, slow, or not available at all? Pier-Yves Lessard approaches that tension with a practical tour of debugging by instrumentation, using Scrutiny Debugger as the anchor. The session mixes architecture, a live STM32F411 Discovery board demo, and concrete integration details from a C firmware. It also shows how the same setup can support scripts, a GUI, and embedded graph capture while being independent from the silicon vendor environment. This is useful for embedded engineers who need better observability on real hardware, especially when classic stepping and halt-based debugging are a poor fit.

Who will benefit the most from this presentation

• Embedded firmware engineers who need visibility into a running system without relying on a debug probe
• Developers working on real-time control software, especially when pauses or stepping would disturb behavior
• Test engineers who want to automate observations and capture data from live devices
• Teams using baremetal MCUs who need a lightweight way to expose internal state
• Engineers evaluating instrumentation-based debugging for production-like hardware setups

What you need to know

You will get more from the session if you are comfortable with:

• Basic embedded firmware concepts, including main loops, tasks, and peripheral I/O
• Debugging workflows on microcontrollers, especially probe-based inspection
• Reading simple C firmware and build-system changes
• Common embedded transports such as serial, USB virtual serial, CAN, or Ethernet

Glossary (terms used in this talk)

• Handshake: A coordination mechanism used by digital interfaces to transfer data when both producer and consumer are ready. In streaming hardware, handshakes often use valid and ready signals to manage flow control.
• Ring buffer: A circular buffer used to pass streaming data efficiently between producer and consumer stages.
• Instrumentation-based debugging: A debugging approach that exposes internal state through firmware-controlled data exchange over an existing device interface. It trades direct hardware inspection for runtime observability that depends on code running on the target.
• Firmware description file (SFD): A metadata file that describes a firmware image, including memory layout and symbol information needed to locate variables on a connected device. Tooling can use it to map runtime data back to meaningful names and addresses.
• Alias: A virtual variable name that points to a real variable and can optionally apply transformations such as scaling or offset. Aliases provide a stable, human-friendly path for external tools and scripts.
• Half-duplex: A communication mode where data flows in only one direction at a time on a shared channel. The direction switches as needed, but simultaneous two-way transmission is not possible.
• Embedded graph: A device-side capture mechanism where the firmware samples selected variables into an in-memory ring buffer and, upon a software trigger, uploads the buffered acquisition to the server for analysis.
• Firmware ID: A unique identifier embedded in the firmware and associated SFD that allows the server to match a connected device to its firmware description and metadata.

Toolbox (mentioned in this talk)

• GitHub: A web-based platform for hosting Git repositories and collaborating on software development.
• CMake: A cross-platform build system generator that helps define and drive builds across different compilers, IDEs, and platforms.
• STM32: A family of microcontrollers commonly used for embedded control and hardware interfacing. They are often used to drive peripherals such as LEDs, motors, and sensors.
• Scrutiny Debugger: An open source instrumentation and test framework for embedded systems. It provides a server, client tools, and firmware-side support for reading variables and capturing runtime data.
• STM32F411 Discovery: An STM32 development board used for embedded prototyping and demonstration. It provides a convenient platform for testing firmware, peripherals, and instrumentation workflows on real hardware.
• Arduino Mega: A microcontroller development board often used for prototyping and small embedded experiments. It is commonly paired with sensors and simple serial-based tooling.

Final thoughts

Practical and demo-driven, this session gives you a working mental model for turning firmware into something you can observe, script against, and test more confidently. The value is less about abstract theory and more about seeing how an embedded tooling stack fits together from target to server to client. It will help firmware developers, test engineers, and anyone wrestling with hard-to-reproduce behavior on live hardware. The tone stays grounded throughout, with an engineer’s bias toward things that can actually be built and used.

This overview is AI-generated from the session transcript. Spot an issue? Let us know.

Edited by the speaker on May 6, 2026.