Home > On-Demand Archives > Talks >

Test-Driven Development for Embedded Software

James Grenning - Watch Now - Duration: 56:58

You've heard about Test-Driven Development but have never tried it or don't quite get it. Test-Driven Development is an important design and problem solving technique that helps software developers improve product quality and the quality of their life. How? By preventing defects, protecting your code from unintended consequences, and giving you warning when your design starts to deteriorate.

In this presentation James describes the problems addressed by TDD. He will define TDD and show you a short example of TDD. He'll tell you some of the benefits you can expect from TDD as well as the challenges of applying TDD to embedded C and C++.

James Grenning is inviting you to a scheduled Zoom meeting.

Topic: TDD for Embedded Happy Hour Embedded Online Conference
Time: May 20, 2020 03:00 PM Eastern Time (US and Canada)
Every day, 2 occurrence(s)
May 20, 2020 03:00 PM
May 21, 2020 03:00 PM
Please download and import the following iCalendar (.ics) files to your calendar system.
Daily: https://us02web.zoom.us/meeting/tZYpce2ppjMrEtMuPJuyVD1AFYDeKiBVi3vc/ics?icsToken=98tyKuGqqj0uG9ydsRGARpwQBo_oLOvxiFxcj7dwiS_PFjllRlLXENtmN5l2Mu7Z

Join Zoom Meeting
https://us02web.zoom.us/j/82464188187?pwd=UEloazJyTklpZlFTRFFxVjNPdmlUQT09

Meeting ID: 824 6418 8187
Password: 031453
One tap mobile
+13017158592,,82464188187#,,1#,031453# US (Germantown)
+13126266799,,82464188187#,,1#,031453# US (Chicago)

Dial by your location
+1 301 715 8592 US (Germantown)
+1 312 626 6799 US (Chicago)
+1 646 558 8656 US (New York)
+1 253 215 8782 US (Tacoma)
+1 346 248 7799 US (Houston)
+1 669 900 6833 US (San Jose)
Meeting ID: 824 6418 8187
Password: 031453
Find your local number: https://us02web.zoom.us/u/kflJGFuSy

M↓ MARKDOWN HELP
italicssurround text with
*asterisks*
boldsurround text with
**two asterisks**
hyperlink
[hyperlink](https://example.com)
or just a bare URL
code
surround text with
`backticks`
strikethroughsurround text with
~~two tilde characters~~
quote
prefix with
>

Doini
Score: 0 | 4 years ago | 1 reply

Thank you very much, it is very useful!

jwgrenningSpeaker
Score: 0 | 4 years ago | no reply

Cheers!

Cesar_Santos
Score: 0 | 4 years ago | 1 reply

Great presentation!!! I have always heard of your book as the main reference in embedded domain, and after this presentation, it's now definitely on my shopping list :)
Regarding integration/system-level tests, is it possible to create one that can validate timing constraints? I was thinking if there's some framework or recommended ways to validate in-between modules communication (which can be logical/software level only, or maybe even in-between chips), which can be critical to some systems.
Thanks in advance.

jwgrenningSpeaker
Score: 0 | 4 years ago | no reply

Generally, TDD is about behavior. Timing is an orthogonal issue. You can probably figure out how to instrument your product code with a way to monitor timing. I expect these could be expensive tests. I'm just guessing. To start down that path, growing your TDD and unit test skills will help.

Elias
Score: 0 | 4 years ago | 1 reply

Hi James, really enjoyed your presentation and style! Right now I'm doing unit testing and debugging of code, passed by an independent developing team which is no longer available... the waterfall is right over me! wished they'd have followed TDD! Thank you very much for this introduction on the topic.

jwgrenningSpeaker
Score: 1 | 4 years ago | no reply

Elias, I hope you survive the drenching! You can add unit tests to legacy code as a way to debug it. It's not easy but might be worth it for you. You can find my legacy code recipe and specific solutions to problems I've run into here https://wingman-sw.com/articles/tdd-legacy-c.
When you come up for air, ping me to help you turn the waterfall into a fast-moving stream.

Eldot
Score: 0 | 4 years ago | 1 reply

Thanks for the lecture James, your style is very entertaining. Your slide "Misconception and Sustainability" reminded me of a project in the mid 90's when the launch of our new system was delayed by 1 year because we were using TLD. After 2 years to develop the hardware, the embedded team took 6 months to test and fix their firmware. Finally the Windows team got to try their code against the system, but found numerous mismatches in the understanding of the design specification across the hardware, firmware and software teams, so the launch was delayed 1 whole year rewriting code, trying the get the thing to work. For me, TDD reduces the surprise of integration test failures like this, meaning that the launch date of the product can be more accurately forecast and the development team not face massive time pressure from the management at the end of the project. Suffice to say that the company closed 5 years later due to lost market share.

jwgrenningSpeaker
Score: 0 | 4 years ago | no reply

Thanks for your comment Eldot.
It's nice that there now is another way!

Rebeca
Score: 0 | 4 years ago | 1 reply

Thanks for the lecture! I like the insight of running unit tests off-target, I will try to implement it in my company. We are getting into test-driven development but we run the tests only in target, which actually takes dozen minutes to build. So I felt like test-driven development was just too slow. I believe that running off-target will help a lot!
A question - Would you say that simply mocking the platform-dependent libraries in a unit test is better than using a simulator?

jwgrenningSpeaker
Score: 0 | 4 years ago | no reply

HI Rebeca, I thought I replied to this a few weeks ago. Sorry to not press submit :-)
Mocking and simulators are different things and are both helpful. Mocks, spies, fakes are great for driving your code through its different paths during unit tests and some integration tests. A simulation would probably be most helpful during integration and load testing.

I'd be happy to discuss it further. James

Otzen
Score: 0 | 4 years ago | 1 reply

Thanks for the tak.
Just wanted to add a comment, I recently started to prepare a setup for introducing unit tests on our legacy embedded codebase. To make the unit tests run on PC, I needed to compile the code using a host compiler.
An unexpected learning here was that just the action of exposing your code to another compiler, made potential bugs pop up. Different compilers complaint on different things, each enabling new insights.

jwgrenningSpeaker
Score: 0 | 4 years ago | no reply

You are welcome. Thanks for watching and your comment. I'll agree and add that making your code work on two compilers means that the next compiler port even easier.

jwgrenningSpeaker
Score: 1 | 4 years ago | no reply

Hey all you people interested in TDD. If you want to be immersed in it for a few days, consider my web delivered training. https://wingman-sw.com/training/remote-delivered-tdd
It could change your life! Your's would not be the first.

Cheers, James

Yunus
Score: 0 | 4 years ago | 1 reply

Hi. Thanks for great presentation.
What are your opinions if you are developing something new. I mean, the thing is you have never done, the thing you researched and decided to develop, the thing that doesn't have been experienced by you before.
Since in that kind of work everything has possibility to be thougt incorrectly, to be done in another or more efficient way etc, making TDD tests along with code might double your work before you rewrite everything (which is very possible). Do you suggest TDD even for that situations? What do you suggest

jwgrenningSpeaker
Score: 0 | 4 years ago | no reply

Hi Yunus
I have a presentation about your question, recorded at Agile 2019 called "Deep Stack – Tracer Bullets from ADC to Browser". Here's the link:

https://www.agilealliance.org/resources/sessions/deep-stack-tracer-bullets-from-adc-to-browser/

It's about a green field product I was working on and how and where TDD fit in. The short answer is that while I am discovering how the system works, I don't write tests for the code at the boundary of the unknowns (like HW, or some external API). But I am testing my understanding though little prototypes and experiments. Once my understanding solidifies, I can test drive where it helps. Which is most the code if you design for test.

kii
Score: 0 | 4 years ago | no reply

This was brilliant

zerorunn3r
Score: 0 | 4 years ago | no reply

Excellent! Very impressed with the style of the presentation and the message. Thanks!

Mina
Score: 0 | 4 years ago | 1 reply

This was a very great helpful talk, thank you James,
i enjoy the way you apply the TDD on Embedded systems, in the beginning i though TDD is not for Embedded Systems but now i believe it will help a lot.
Can we apply the same to a High Safety Critical Applications? were we almost write all the Block Diagrams, functions, interfaces,... as Pseudo Code(or higher level language) then test it then Coding is the last thing to do.

jwgrenningSpeaker
Score: 0 | 4 years ago | no reply

Hi Mina
I don't see why you would not be able to use TDD for safety critical. TDD is really about being careful, and making sure your code does what you think it is doing. You could start without changing your design process, and then grow the modules you specify in diagrams with TDD. You will probably evolve your design process as you grow your skills in TDD and see how some of the earlier design ideas were prematurely decided.

Let me ask, when you finally get to coding in your process, do you change and evolve the design ideas as the reality of code becomes evident? The test-driven approach is realistic in that it assume that we will change design ideas (and system needs) as we get into solving system design problems.

If you want to talk further, contact me. https://wingman-sw.com/contact-us

foz
Score: 0 | 4 years ago | 1 reply

James! Great stuff. I just thought I would say debug statements in context of logging is not necessarily an exact swap with test driven code. Often times Logging is important for multithreaded code, reporting calculated values, or to provide a tool for a non-sw engineers to do first pass debug through your application (say a service organization). In other words, sometimes logging (which sadly some people use entirely as their debug method) actually enables other people with a tool to inspect behavior.

jwgrenningSpeaker
Score: 0 | 4 years ago | 1 reply

Hi Foz

The TDD concern I was addressing, that I hear basically from every group I teach TDD, is that TDD means there are too many lines of code to maintain.

My comments about comparing LOC for test and LOC for debug is in part to get people thinking about the alternative and how they currently do not even notice the amount of code dedicated to debugging. I am not saying logging is not needed. Application/product usage logging is very valuable. The logging you mention is not what I am talking about.
I want people to self-reflect on their work and code.

I hope this statement gets people thinking:

People object to code added proactively to prevent defects, but accept without concern all the debug statements added to code reactively to respond to defects.

Thanks for the comment!
James

foz
Score: 0 | 4 years ago | 1 reply

"People object to code added proactively to prevent defects, but accept without concern all the debug statements added to code reactively to respond to defects."

Yeah I like that. Thanks for your hard work! I used to teach at CU Boulder ECE, I just submitted your book to their curriculum as the foundation for embedded test and debug in their curriculum reform.

jwgrenningSpeaker
Score: 0 | 4 years ago | no reply

Thanks Foz!

SimonSmith
Score: 0 | 4 years ago | 1 reply

I enjoyed reading the book thanks. It was the first time I’ve heard about TDD in the context of embedded software. I first used it on a C# web app and it was a real eye opener.

jwgrenningSpeaker
Score: 0 | 4 years ago | no reply

Thanks for the comment.

RayS
Score: 0 | 4 years ago | 1 reply

Great presentation and explanation of TDD.
The TDD "microcycle" seems to encourage incremental coding, at the lowest level to solve the current test case. Do you find that the higher level design view gets lost and that developers implement diverging approaches, for example one interface copies data while another interface passes pointers. Do you have strategies to prevent that?

jwgrenningSpeaker
Score: 0 | 4 years ago | no reply

Hi RayS
I encourage developers to have a vision of where they are going. Kent Beck, the creator of Extreme Programming and TDD calls it a system metaphor. I find the term architectural vision to be helpful. Call it high level design if you like.

Also I encourage collaborative work, including

  • common language around communicating about code problems (Fowlers code smells)
  • common language about design (Robert Martin's SOLID for starters)
  • agreed coding standard
  • collaboration in real time with pair programming, mob programming or very frequent reviews
HR
Score: 0 | 4 years ago | 1 reply

Good stuff James! Really enjoyed your book, although might need to read it again to refresh my memory :)

jwgrenningSpeaker
Score: 0 | 4 years ago | no reply

Thanks!

Ilias
Score: 0 | 4 years ago | 1 reply

Excellent Book and methodology. Once there never look back (almost...)

jwgrenningSpeaker
Score: 0 | 4 years ago | no reply

Thanks!

Miro
Score: 1 | 4 years ago | 1 reply

One aspect of TDD that I enjoy the most (and it should be emphasized more!) is the sheer fun of programming that way. It's becoming like an addictive game. You make progress! You actually exercise your code. Contrast this with DLP (Debug-Later Programming). It's getting boring very quickly. You lose interest...

jwgrenningSpeaker
Score: 1 | 4 years ago | no reply

HI Miro
Thanks for reminding me of this essential part of TDD that makes it sustainable

TDD is Fun!

I stress to people new to TDD that this value proposition:

What if you could do more of what you like and less of what you don't like?

TDD is Fun, saves time and money. Who's not up for that?

It does take some time to learn.

Pete
Score: 2 | 4 years ago | 1 reply

Great presentation. I'm early in my TDD career, but I am starting to get the hang of it. I recently had the chance to do a SW review where we added a feature to some 'high value' technology for our business that is super hard to test. In that review I showed the final integration test of the feature in Cpputest before we transitioned to HW testing. It was an eye opener for several folks at our company to see the technology running outside of its hardware in a test rig.

Oh, that is a great assortment of T-shirts.

jwgrenningSpeaker
Score: 0 | 4 years ago | no reply

Thanks Pete. I'd love to hear more about your story. I collect Stories from the Field on my website and invite you to write one!

Jeremy
Score: 1 | 4 years ago | 1 reply

Excellent talk James!

jwgrenningSpeaker
Score: 0 | 4 years ago | no reply

Thanks Jeremy.

Score: 2 | 4 years ago | 1 reply

Thanks for producing the most entertaining video of the whole conference. I appreciate the extra time and effort you put it to jazz up the presentation. Your example of delivering the cracked cup holding by a string, is a really nice metaphor. I am a fan of TDD, being on the validation side, and promote it when I have the chance. But I am consistently placed under the roaring, uncontrolled waterfall. For embedded systems, in which various subsystems are from different years, it is admittedly difficult to get a consensus for TDD. Getting all the timings right, and the functionalities right, every time, with limited development and test time, and resources, remains a challenge.

jwgrenningSpeaker
Score: 0 | 4 years ago | no reply

Thanks Jean! Hopefully TDD can help you out from under the waterfall!

MatthewEshleman
Score: 0 | 4 years ago | 1 reply

I upvoted JeanGoulet's comment. What an amazing level of effort. Thank you very much!

jwgrenningSpeaker
Score: 0 | 4 years ago | no reply

Thanks Matthew

Score: 0 | 4 years ago | 1 reply

very nice presentation. Very similar to what I am teaching myself to learners of Quebec,Canada. An pros to using TDD that I would have liked to hear more is the fact that this technique drives the project's functions to be easier to use. Since you write the test first, you decide here and now how is the easiest way to call a specific function or to obtain a specific behavior. You then have the hard task of dealing with this "ease of use" requirement inside said function. DLP programmers tend instead to code functions the easiest way they find, leading into complex calls, parameters hard to provide, etc.

jwgrenningSpeaker
Score: 0 | 4 years ago | no reply

I totally agree. There is a big focus on interface in TDD. My article, TDD Guided by ZOMBIES , reinforces that idea.
Thanks for the comment.

Hammarbytp
Score: 0 | 4 years ago | 1 reply

Excellent talk. Question - can unit tests be used for legacy code, i'e added later. How does affect the process

jwgrenningSpeaker
Score: 0 | 4 years ago | no reply

The test-drive developer approaches legacy code carefully. They understand their vulnerabilities. Code is easy to break and hard to keep working. We follow the principle:

It's easier to keep a system working than to fix it after you break it.

In the context of changing legacy code, you'll do many of the things you currently do, like dig through the code to try to understand the code. Before changing code, we'll usually bring the file, ideally untouched, into a unit test harness. (sometime some minor organizations changes are made, not logic changes).
Say you are fixing a bug and you suspect a particular function...

  • Call the function from a test case
  • Write a test to reveal the bug
  • Write some more tests, as bugs nest together and you do not want any unintended side effects. Also the tests help you confirm your understanding.
  • Fix the bug.
  • Do some improvement to the code, now that you have it under test.
    Here is an article on the topic TDD How-to: Get your Legacy C Into a Test Harness
jwgrenningSpeaker
Score: 0 | 4 years ago | no reply

Thank you Jacob

Jacob_Beningo
Score: 3 | 4 years ago | no reply

The slides are now posted! Sorry about that!

jwgrenningSpeaker
Score: 0 | 4 years ago | no reply

Please join the zoom meeting

OUR SPONSORS