Test Your State Machine Monstrosities
📢NOTE! See bottom of abstract for useful info.
Let’s efficiently unit test and integration test our state machine monstrosities! The general approach works with multiple unit testing frameworks, but Google Test has some distinct advantages, so we’ll use that. We’ll laugh, we’ll cry, and in the end, we’ll achieve full hierarchical state transition and behavior coverage in an easy-to-maintain manner.
The general concept works with any type of state machine (hand written, code generated, C, C++, Python…).
Topics include:
- “Hand of God” vs “natural flow” testing — pros and cons
- Unit testing and integration testing approaches
- Testing embedded C state machines with C++
- Google Test tips: fixtures, fakes, mocks, scoped traces, and more
Github Repo
Highly recommend cloning the example github repo to try out the concepts here: https://github.com/adamfk/eoc-2026
Video Timeline:
- 1:52 - working example
- 2:40 - test pyramid
- 3:30 - off target tests diagram
- 4:30 - off target test benefits
- 5:13 - unit test types
- 6:25 - unit testing vs integration testing
- 8:35 - composition pattern
- 10:02 - faking/mocking - link time substitution
- 10:57 - faking/mocking - runtime substitution
- 11:54 - our first unit test with a FakeHal, real code, some google test tips and tricks
- 26:28 - fsm testing intro
- 27:23 - Hand of God testing
- 42:38 - Natural Flow testing
- 49:36 - Tips/tricks, advanced topics, mocks, more info
Thanks! Feedback welcome.
What this presentation is about and why it matters
How do you test a state machine when forcing states, nested hierarchy, and hidden entry behavior can make a test either very fast or very misleading? Adam Fraser-Kruck tackles that tension with a practical walkthrough rooted in embedded C, C++, CMake, and Google Test, using a lights controller and HAL as the running example. The session compares hand-of-god testing with natural-flow testing, then shows how test fixtures and helpers change the day-to-day work of testing more complex machines. It is especially relevant if your code has stateful behavior, layered dependencies, or awkward test seams.
Who will benefit the most from this presentation
- Embedded software developer working on stateful control logic, especially if state transitions are hard to reach through the UI.
- Test engineer or QA-minded developer who wants faster off-target tests without losing confidence.
- Firmware engineer maintaining C code with HAL boundaries and looking for safer seams for fakes or mocks.
- Developer using hierarchical or generated state machines and needing a scalable test strategy.
- Team lead deciding how much testing belongs in unit tests versus integration tests for embedded work.
What you need to know
A basic working knowledge of embedded testing helps, along with familiarity with state machines and the idea of fakes or mocks. The talk uses a few concrete tools and conventions:
- C and C++ source files mixed in one project
- CMake-based builds
- Google Test test cases and fixtures
- Common testing terms like unit test, integration test, fake, and mock
- Reading simple state diagrams and transitions
Glossary (terms used in this talk)
- Test double: A stand-in used during testing instead of a real collaborator. Test doubles include fakes, mocks, stubs, and similar substitutes, each trading realism, setup cost, and assertion style in different ways.
- Off-target testing: Tests that run on a host machine rather than on the embedded target. These tests usually run faster, are easier to automate, and can be debugged with desktop tooling.
- Hand-of-god testing: A testing approach that forces a system directly into an internal state instead of reaching it through normal behavior. It can make edge cases easier to cover, but it also couples tests tightly to implementation details.
- Natural-flow testing: A testing approach that reaches a state by driving the system through its normal transitions. It tends to reflect real use more closely and can reveal problems in the transition path, not just in the destination state.
- Entry behavior: Work performed automatically when a state is entered. Entry behavior matters in tests because forcing a state may skip the code that normally runs on entry.
- Fake: A test double that provides a working implementation of a dependency for tests, often simpler than the real implementation and used to observe state or outputs rather than assert call order. Fakes are typically used to make tests more robust and easier to refactor than strict mocks.
- Mock: A test double that records expectations about calls (including order and arguments) and verifies those expectations during test teardown; useful when the order of interactions is critical but can lead to brittle tests if overused.
- Link-time substitution: A technique for replacing dependencies in tests by providing alternative implementations at link time so that the test executable links against fakes instead of the real components, commonly used in C.
Toolbox (mentioned in this talk)
- GitHub: A web-based platform for hosting Git repositories and collaborating on software development.
- PlantUML: A text-based diagramming tool for generating UML and other technical diagrams from plain text descriptions.
- Visual Studio Code: A lightweight source code editor with broad language support and extension-based integration. It is often used for embedded development and can pair well with containerized workflows.
- Google Test: A unit testing framework for C++ that supports assertions, fixtures, and structured test suites. It is commonly used to automate verification of library and application code.
- CMake: A cross-platform build system generator that helps define and drive builds across different compilers, IDEs, and platforms.
- Statesmith: A state machine code generation tool for embedded software and related projects. It can generate code from a state machine definition and support workflows for testing generated machines.
Final thoughts
Practical and example-driven, this talk gives you a better way to think about testing stateful embedded code, especially when the machine is large enough that simple unit tests stop being enough. You will come away with a clearer mental model for where test seams live, how entry behavior changes the game, and why test helpers matter once state counts start growing. It should be especially useful for embedded developers who work in C or C++ and want tests that stay readable as the design evolves. The overall spirit is pragmatic confidence, not testing theater.
This overview is AI-generated from the session transcript. Spot an issue? Let us know.
Thanks Steve! Really appreciate the feedback and tips :)
I'm going to try out your suggestion of "Test fixtures per state". I think it could significantly help with test naming and organization. My example repo was a bit rushed there.
Regarding SCOPED_TRACE(): I added another flavor/option to the example repo. V2_ControlTests3.cpp provides the best stack tracing, but is a bit slower to write.
Thank you for the talk, very interesting subject, very well presented and organized. I'd like to share a couple of thoughts -
How to test for an FSM to truly return to a specific state? The labelled state is usually just a part of the real (hidden) state. A state may contain counters or references to resources, and other information that can be taken into account when the state code decides whether to switch to a new (labelled) state or not.
Black box testing doesn't help since returning to the behavior of state X, after transitioning through other states, doesn't prove that the machine is really in the same state (i.e. next closed path transitioning through states may sport a different behavior).
White box testing may result too coupled to the implementation.
In a past project, I added a method to FSMs to hand out an opaque read-only memory area where all the state was stored, so that the test could memcmp those areas to assert that the machine was again in the very same state. Not an elegant solution, nor perfect for all cases, but it works.
Another thought is about FSM implementation. If you model each state with a function that receives the input (and any additional state data if needed), and returns the next state (and if needed, next state data), then testing states becomes easier - you can actually test all the conditions without requiring the FSM to evolve through every possible path. Still quite coupled to the implementation, but straightforward to test.
Great points Max :)
You raise a very important point about testing the "extended state" (which includes state, data, counters, ...). I agree, it is impractical to Black box test this with certainty. Especially if a FSM has multiple paths & loops as there are infinite permutations.
Slides 34 - 40 have a related example and advice on "Path Independence". I added a bit more context & speaker notes to this section after you posted your question. TLDR: If possible, I prefer to have "extended state" only modified by the FSM (and not external code). Use state entry/exit behaviors to consistently and clearly set "extended state".
Your white/grey box idea for asserting "extended state" (which includes data, counters, ...) is a good one, but watch out for memcmp() on a struct that may have padding between fields. I'm sure you are aware, but I'll add some more detail here for other readers. This memcmp() approach will probably work fine most of the time, but it isn't always guaranteed. Here's a godbolt example of memcmp() randomly failing to compare struct fields because of padding.
Unfortunately, I'm not aware of a "perfect" way of asserting "extended state". Every solution has pros/cons. If you do safer field by field comparisons, then you need to make sure that you don't forget to update that code when a new field is added. Static assertions on sizeof and offsetof can often catch structure evolutions. Structure packing removes padding but can make code slower, makes some assignment operations non-atomic, and can lead to undefined behavior/hard fault with pointers to misaligned member (Cortex M0)...
I also (mostly) agree with your last point about implementation. If your FSM is essentially a "pure function" and doesn't call any external methods (side effects), it can be easier to test. This works well for data oriented FSMs like parsers. That said, a "pure function" FSM may still rely on an earlier state to setup context data appropriately (like allocating memory).
Adam
Thanks for the talk.
We are currently using ceedling and the fake function framework, mainly because it just works®.
It seemed most of your talk would apply to ceedling too; I use 'hand of god' a lot, tbh, and thought designing unit tests to look at a single function per unit test was best practice.
Would you know of any pitfalls around bringing the 'natural flow' and integration test methods into a ceedling/fff setup?
Hi Scott,
"Hand of God" testing isn't necessarily bad. It just has some gotchas and limitations to be aware of. If it is working well for you and your team (and you are aware of what to watch out for), keep using it.
I'm going to use this opportunity to share some info that I wanted to add to the presentation, but it was already at the time limit. I hope you don't mind :)
<fatal-vs-non-fatal-tests>
One feature I really like in ceedling (compared to gtest) is that when an assertion fails, it stops the test immediately. This "Fail Fast" behavior is often quite desirable in embedded systems unit testing.
gtest specifically supports non-fatal assertions (EXPECT vs ASSERT macros). These non-fatal assertions are really helpful if your compile/test times are long like 30 minutes. You want to collect as much failure information as possible in a single compile/test run. That said, I think this is mostly useful for massive C++ code bases (like chromium).
I've never worked in true safety critical, but I have worked in off-highway vehicles and many products would have heavy unit/integration testing. Hundreds of tests. These C++ test suites (testing C embedded code) compiled and ran in under a minute or so.
Over 95% of the time, I just want my tests to "Fail Fast". Accumulating multiple failures in a single test often just adds more "noise" that you need to sift through and understand. Interestingly, cpputest explicitly chose to stick with just fatal assertions.
</fatal-vs-non-fatal-tests>
The main downside to 'natural flow' / integration testing in low level languages like C/C++ (that don't provide stack traces in test assertion failures) is that diagnosing test failures can be more difficult inside of test helpers. And you need to use test helpers to stay sane.
If you don't care about diagnosing test failures from CI/CD test logs, you can simply just debug the failing test to understand the call stack.
Or you can use a bit of clever printf formatting to print stack trace like information. Here's an example.
Hope this helps,
Adam
P.S. If you have more related ceedling tips, could you share them here? I'd love to read them.
Hi Adam, this is good material, the trick to do "fakes" over mocks is great, even if we are doing everything in C for the on target code I can see the advantage of using C++ for unit tests. Mocks can get annoying over time and become harder to maintain . Nice talk
thanks :)
This is a fantastic presentation! State machines can truly be a monstrosity of implementation, bugs, and maintenance if done poorly. This brings a lot of sanity to them. Great combined with Miro Samek's material.
The "hand of God" method is very seductive and seems easy, but rapidly risks getting very brittle. I've seen all the variants of it that Adam shows and agree with the pros and cons listed here. Sometimes a slight improvement is making "SetStateToXYZ" functions that are public along with a "GetState" function, with the expectation that only the test code and the real client code will call them. They are public symbols that any other code could call, but as a practical matter nothing else in the codebase does. But depending on the complexity of state transitions and the set of state variables (counters, timers, buffer contents, etc. in addition to the explicit state value), this can also get messy and brittle rapidly.
The "natural flow" method is exactly my preferred method for FSM testing. Helpers are critical to it. Without them the test code becomes a monstrosity itself, but with them, it is truly natural, with the nested function calls chaining from state to state. The helpers make the setup for each test super-DRY. Test fixtures per state are the way I like to organize this.
I really like the SCOPED_TRACE_FUNC() to see where the helpers were called. I'll have to look into the details of that.
A failure due to a helper needing to be updated can result in a hard-to-read cascade of output. I've accepted that changes to the FSM mean I need to change the helpers (because that is a change in behavior) and that can be a bit messy along the way. The good news is that by using helpers, the change is generally just isolated to a single place, and then the whole test suite is back to working. That's a situation that can go from multiple failing tests to all working with just a single line change in a helper.
Adam also does an excellent job of talking about using mocks vs. fakes. Mocks have their place, such as verifying specific sequence, but most of the time are just over-specification that make the tests extremely brittle. I've worked on a large codebase that was all mocks all the time, and the slightest change to implementation meant multiple failing tests. Then I had to reverse-engineer the mocking in all those tests to understand how to fix them. That creates a huge maintenance burden.
My favorite alternative to mocks is what Adam calls "recording fakes", what Grenning calls "spies". Something that captures output or side effects in a way that the test can examine them and assert expected results.