Home > On-Demand Archives > Talks >

Testing of Event-Driven Embedded Software with Python

Miro Samek - Watch Now - Duration: 41:04

Testing of Event-Driven Embedded Software with Python
Miro Samek

Testing event-driven software typically involves injecting events to the system and verifying that the software reacts by executing the expected sequence of actions. These actions might be quite complex. For example, a single transition in a hierarchical state machine might trigger several exit actions from the source state configuration, followed by entry actions and nested initial transitions into the target state configuration. The actions themselves might involve interactions with multiple hardware components, such as toggling GPIOs, sending data through various interfaces, etc.

In traditional unit testing frameworks, the verification of such potentially complex interactions typically requires applying mock objects as the test doubles for the various hardware components. But even though generating mocks can be automated, it's difficult to verify sequences of expected interactions among multiple mocks (multiple hardware components). Also, working with mocks reverses the natural sequencing, because the expectations must be specified before the actual call to the CUT (code under test).

This session will show a more intuitive approach, conceptually similar to "debugging with printf", where you instrument the code with printf statements to output the software trace information. You then run the CUT with a controlled set of inputs, and examine the produced trace to determine whether the CUT operates correctly. The main differences from using actual printfs will be: (1) that the much more efficient software tracing mechanism will be used instead and (2) that both generating the inputs and the checking of the trace output will be automated.

This approach has many interesting implications:

  • It separates the execution of the CUT from checking of the "test assertions". The embedded target is concerned only with running a test fixture that calls the CUT and produces the trace, but it does not check the test expectations.
  • Checking the generated trace output against the expectations is performed separately on the host computer and can be done in a different programming language (e.g., Python) than the CUT (C or C++).
  • Virtually all test doubles can be implemented as simple "spy" test doubles without the need to applying mocks or other complex test doubles. Verifying trace output from multiple such "spy" test doubles is trivial.
  • The testing process follows the natural sequencing where the generating of inputs to the CUT is naturally followed by the expectations of the performed actions. 

This session will utilize hands-on demonstrations of the QUTest (pronounced "cutest") unit testing harness running on both the host computer and STM32-NUCLEO board. Specifically, you will see a comparison between the traditional approach with a mock test double and the QUTest approach with software tracing and a "spy" test double.

italicssurround text with
boldsurround text with
**two asterisks**
or just a bare URL
surround text with
strikethroughsurround text with
~~two tilde characters~~
prefix with

Score: 0 | 9 months ago | 1 reply

Are QSpy and QUTest part of the QP/C licensing model ? or are they selfcontained tools provided with their own licenses ?

Score: 0 | 9 months ago | no reply

Yes, QSpy and QUTest use the same licensing model as QP/C and QP/C++, which means that they are available under GPLv3 and potentially under a commercial license from Quantum Leaps. Please note, however, that most of the time you don't ship the test/trace code inside embedded devices. Instead, the tracing and testing occurs in-house, where you can easily comply with the GPL.

Score: 0 | 9 months ago | no reply

If anybody is interested in actually running the examples, all the code presented is available on GitHub:
Specifically, the projects from the presentation are in the sub-directories: unity_strlen, unity_ledbar, and unity_ledbar2.

Score: 1 | 9 months ago | 1 reply

Excellent presentation!

Score: 1 | 9 months ago | no reply

I'm glad that you liked it!