Home > On-Demand Archives > Keynote Presentations >
The Embedded Renaissance: Reimagining How We Build Firmware in a Modern World
Jacob Beningo - Watch Now - EOC 2025 - Duration: 01:44:30

In today’s rapidly evolving embedded landscape, outdated development methods can leave you struggling to hit deadlines, control costs, and maintain quality. Whether you’re leading a team or writing the code yourself, it’s time to modernize your approach, secure your competitive edge, and open new doors for innovation.
This talk presents a proven, 7-step roadmap for transforming your embedded software practices, designed for both strategic decision-makers and hands-on developers. Discover how to establish a metrics-driven scoreboard to measure progress, adopt modern, modular architectures to streamline delivery, and integrate DevOps pipelines that enhance speed and consistency. Learn how to standardize build systems, embrace test-driven development and simulation for earlier feedback, and incorporate AI/ML capabilities that propel you ahead of market trends.
Attendees will walk away from this talk with a clear, actionable roadmap to modernize their embedded software practices—from planning and measurement to delivery and innovation—empowering them to stay ahead in an ever-changing landscape.
Topics Covered in This Talk Include:
- Metrics Scoreboard: Automate a Software Metrics Scoreboard
- Architecture: Adopt Modern Architecture Methodologies
- DevOps: Integrate DevOps Pipelines
- Build Systems: Standardize Your Build Systems
- AI/ML: Leverage AI/ML to Accelerate
- TDD: Embrace Test-Driven Development to Improve Testability
- Simulation: Leverage Simulation Early
Thanks for the comment!
To date, I've not found a lot of tools that are good at this. My own simulation I often just create functional simulators where I write an interface to decouple hardware and then write python or server code that feeds in what the hardware would to test the application code.
At one point I was very excited about Matlab for this, but have found overall the learning curve is too high and the generated code is not human readable or good.
Renode has been interesting for simulation and Embedd for generating quality drivers and interfaces.
I hope that helps, if not, feel free to follow up!
Fantastic presentation Jacob. As a engineer that mainly works with FPGA i find many of the steps hard to implement. I know that many people dont consider writing FPGA code as firmware, beacuse you are basically describing hardware, but thats a discussion for another time. I always try to find ways to make my build proccess cleaner and better but it always ends up being kind of frustrating and its not worth the effort. I wonder if you, or anyone for that matter, has some experience in integrating FPGA in a DevOps enviroment
Thanks!
I think that you could consider the FPGA code as firmware, even though we don't traditionally think of it as such.
Unfortunately, I don't have much experience with FPGAs. It's always been of interest but I've not had the bandwidth.
Adam Taylor would be a good person to ask on that front. He works exclusively with them.
As always a great presentation Jacob! Thank you so much! I always learn so much from you!
Thanks!
Living in Denmark, I see most of these presentations on-demand. This one is really recommendable, and in many ways resonate with my own mindset. For instance I recently checked out Zephyr, which is referred to a lot in this conference, and I was also impressed with "west". You can adapt the Zephyr build-concepts even if you don't use Zephyr.
The idea of "taking a step back" and look at what you are really trying to accomplish with your architecture is great. Love it!
Regarding your point on simulation I think it is worthwhile to mention that when using embedded Linux, you can test soooo much on a PC or a Raspberry PI - before you get your actual hardware. In a project I observed that developers used the PC - also after we had enough HW, simply because of the shorter development cycle with the better tools. And - as you say - you need to emphasize to also test with the real HW. You can also use the real HW as gateway to the physical world, and use sockets to develop your application on PC.
Finally, your slides says Test-Driven, but you moderate it to "focus on the application testing" - which I thing is right. I recently saw James Coplien presenting "Why most unit testing is waste" - obviously provocative, but not entirely wrong. Thanks!
BR Klaus Elk
Thanks! I'm glad to hear that you are enjoying the conference so far!
Thanks for sharing your experiences! Great observations!
13:44:09 From Magpie Embedded to Everyone: Some vendors have started generating CMake files in their SDKs/IDEs which is a big step forwards! Although more would be better, such as a containerised build / CI build scripts. 13:49:51 From Jim.Mrowca to Everyone: Jacob: what advantages does Zephyr RTOS have over AWS' FreeRTOS? I do not think FreeRTOS will be going away soon. 13:51:06 From davyb to Everyone: Single super loop. 13:51:08 From Magpie Embedded to Everyone: Superloop!? 13:51:09 From Eric Habets to Everyone: ARM? 13:51:11 From Todd to Everyone: Zephyr has strong vendor support 13:51:14 From AEncarnacao to Everyone: ARM 13:51:15 From Viktor to Everyone: Spagetti code 13:51:15 From ryan torvik to Everyone: node 13:51:16 From Keith J to Everyone: superloop 13:51:18 From brian to Everyone: foreground background 13:51:32 From ryan torvik to Everyone: Great image... 13:51:35 From Jim.Mrowca to Everyone: Also, with Torvalds last year declaring Linux as and embedded RTOS - will we still be seeing different RTOSs being developed? 13:53:19 From James Grenning to Everyone: Agileists thinka bout architecutre too! 13:53:44 From dteubl to Everyone: With the developer turnaround on most projects, most systems will - unfortunatelly - develop itself to the Big Ball of Mud. --- personal take. 13:53:45 From Todd to Everyone: I thought the embedded Web Assembly made a lot of sense 14:01:23 From Todd to Everyone: 👍code reviews 14:04:13 From Harshad to Everyone: Curious to see how CI can do code reviews 🤔 14:04:33 From kortej1 to Everyone: With regard to documenting design - classes, message sequence charts, etc... what tools are people using? Also what tools are being used for enforcement of established coding standards on the CI server? 14:05:16 From Patrick Wright to Everyone: Replying to "Curious to see how CI can do code reviews 🤔": I believe pull requests in GitHub/Bitbucket is an example. Require N number of people to review and "sign off" on your changes. 14:05:17 From Todd to Everyone: CI happens after code review and approval AFAIK 14:06:43 From Magpie Embedded to Everyone: CF - Continuous Feedback? 14:07:39 From ryan torvik to Everyone: Replying to "With regard to documenting design - classes, messa...": For coding standards, we run a rust formatter on SAVE. Comes in the dev container we use in vscode 14:07:54 From James Grenning to Everyone: Release to a customer is a business decision. The ability to release at any time means you can make that you can release when you need/have to. 14:08:26 From Jacob (Not Beningo :D) to Everyone: Replying to "With regard to documenting design - classes, messa...": clang-format / clang-tidy (and some related GitHub Actions) 14:10:04 From Daniel Mendez-Lozada to Everyone: Anyone here using GoogleTest? Or what testing framework are you using? 14:10:20 From Tektelic Group to Everyone: ceedling 14:10:32 From Harshad to Everyone: Replying to "Curious to see how CI can do code reviews 🤔": I see, I thought of some AI tool to do code reviews before CI. Curious to see if anyone already uses some AI tool for code reviews. 14:10:44 From Tyler Crawford to Everyone: I am using Ceedling through a Docker container 14:11:36 From James Grenning to Everyone: Fast feedback! 14:11:48 From Jason Mitchell to Everyone: Replying to "Anyone here using GoogleTest? Or what testing fram...": GTest is good for C++ but Unity is better for C. Zephyr also has its own test framework that works pretty well. 14:11:53 From Harshad to Everyone: Replying to "With regard to documenting design - classes, messa...": We use plantuml for creating UMLs 14:11:54 From Jacob (Not Beningo :D) to Everyone: 10% 14:11:56 From Nathan to Everyone: 20% 14:11:56 From Jui Yen to Everyone: For me ceedling on C based, GoogleTest or CppUTest for C++ based 14:11:56 From Patrick Wright to Everyone: Replying to "Curious to see how CI can do code reviews 🤔": I just tried GitHub Copilot's review of C/C++ code in GitHub pull requests the other day. It's okay... 14:12:00 From dteubl to Everyone: <10 14:12:02 From kortej1 to Everyone: 30% 14:12:01 From James Grenning to Everyone: 5% 14:12:03 From Mark to Everyone: 80% if its legacy 14:12:07 From Eric Habets to Everyone: Too much time on debugging ... 14:12:52 From James Grenning to Everyone: casey mute yourself 14:13:17 From Immanuel to Everyone: <10% 14:13:26 From Eric Habets to Everyone: @Casey Sanders please mute your mic 14:14:05 From Patrick Wright to Everyone: My "aha" moment was when I wrote a "simulation" of my firmware before actually deploying it. When It was finally run on the hardware, it just worked! Very little debugging required! 14:14:52 From James Grenning to Everyone: Good advice, start TDD at the app. it has the chance for a long and useful life. 14:15:44 From Daniel Mendez-Lozada to Everyone: Maybe a philosophical debate but: when to run tests on-target vs off-target? 14:15:53 From dteubl to Everyone: Replying to "Release to a customer is a business decision. The...": Indeed. Not having "Sacred Release Day/Week", but releseing on demand and testing features in HIL or at site if needed reduces the overall feedback cycle of the system. And makes us, Devs, more happy :) 14:17:55 From ryan torvik to Everyone: Why write tests? Isn’t the compiler enough of a validator??? 🤪 14:18:56 From dteubl to Everyone: Replying to "Why write tests? Isn’t the compiler enough of a va...": Isn't that why we have the customers?...🤪 14:19:01 From James Grenning to Everyone: Replying to "Maybe a philosophical debate but: when to run test...": One trigger: When you change the adapter between the app and the hardware, retest in HW. When the hardware changes, run on HW. 14:19:31 From Daniel Mendez-Lozada to Everyone: Replying to "Why write tests? Isn’t the compiler enough of a va...": This is sort of the first test: does it compile? 14:20:27 From Patrick Wright to Everyone: Replying to "Maybe a philosophical debate but: when to run test...": If it has to do with the "logic" of the application (what it should do) then use an off-hardware test. 14:20:41 From Magpie Embedded to Everyone: I've also had a situation where hot fixes were being made to the board so each software engineer had slightly different boards. 14:21:12 From Patrick Wright to Everyone: Replying to "Why write tests? Isn’t the compiler enough of a va...": But "does it compile" doesn't actually test your application (architecture), it just tests you knowledge of the language in which you are writing your application. 14:21:13 From Keith J to Everyone: @Magpie Embedded - Yup, happens all the time 14:23:00 From Daniel Mendez-Lozada to Everyone: Replying to "Why write tests? Isn’t the compiler enough of a va...": Right... thinking on a pipeline: unnecessary to run the test suite if the code doesn't even compile 14:24:40 From James Grenning to Everyone: When you integrate, the goal is to only find integration problems. 14:25:12 From ryan torvik to Everyone: Replying to "Why write tests? Isn’t the compiler enough of a va...": I’d argue that when using C/C++ “does it compile” means “it is syntactically correct”. But so much of the language behavior is defined at runtime… … so many weeks of my life I’ll never get back… 14:29:45 From ts to Everyone: For the use of AI, you have to provide context, meaning to upload code to a LMM. Is that considered best practise? What do you think? 14:30:20 From Eric Habets to Everyone: Who writes documentation in LateX? Just curious …. 14:30:25 From kortej1 to Everyone: Replying to "For the use of AI, you have to provide context, me...": Seems like a potential IP issue to me.... 14:30:45 From ryan torvik to Everyone: Replying to "Who writes documentation in LaTeX? Just curious ….": Jacob…… 😛 14:30:47 From Jason Mitchell to Everyone: Replying to "Who writes documentation in LaTeX? Just curious ….": I usually use Doxygen and make it generate LaTeX. 14:30:48 From dteubl to Everyone: Replying to "Who writes documentation in LaTeX? Just curious ….": There are a few of us, 14:31:08 From Patrick Wright to Everyone: Replying to "For the use of AI, you have to provide context, me...": My company uses the enterprise version of Github Copilot which I am pretty sure is trained only on your code for your own model. That way your IP says your own. 14:31:16 From Todd to Everyone: for most LLM's doesn't "uploading code to a LLM" mean now the LLM is going to integrate that code into its general use? 14:31:19 From Eric Habets to Everyone: Replying to "Who writes documentation in LaTeX? Just curious ….": I do … 14:31:27 From dteubl to Everyone: Replying to "Who writes documentation in LaTeX? Just curious ….": However, markdown + pandoc is way easier. and you can do many outputs as well. 14:31:30 From Mark to Everyone: Replying to "Who writes documentation in LaTeX? Just curious ….": @Jason Mitchell I want to try that! 14:31:34 From Jim.Mrowca to Everyone: Replying to "Who writes documentation in LaTeX? Just curious ….": Can AI be cloaked to not compromise IP in the open??? 14:31:55 From Magpie Embedded to Everyone: The Zephyr PR process includes review by an AI which can catch errors. Not sure if it blocks merges but it does provide suggested changes. 14:32:14 From ryan torvik to Everyone: A lot of them have a checkbox that says “use my code for my context, but don’t use it to train your models” 14:33:13 From Mark to Everyone: Replying to "for most LLM's doesn't "uploading code to a LLM" m...": Chat GPT lets you choose. But de we trust it? 14:33:20 From Daniel Mendez-Lozada to Everyone: Imagine getting paid by the written LOC, opened PRs, commits... 14:33:23 From Tim Guite | Magpie Embedded to Everyone: Replying to "A lot of them have a checkbox that says “use my co...": Do you trust the checkbox? 😂 14:34:06 From Mark to Everyone: Replying to "The Zephyr PR process includes review by an AI whi...": I agree. Its nice to have an LLM as peer. 14:34:10 From ts to Everyone: Replying to "For the use of AI, you have to provide context, me...": As a solo dev, this is simply not affordable. So actually I use AI but on a general basis, the results are manually put back in my dev projects (which is sometimes a lot of (wasted) work, but that is what I’m allowed on customer projects). I found no better idea so far. 14:35:29 From dteubl to Everyone: Replying to "Imagine getting paid by the written LOC, opened PR...": didn't that was tried out already in the 80's, 90's? :D 14:35:36 From James Grenning to Everyone: Good job Jacob! 14:36:30 From Mark to Everyone: Replying to "For the use of AI, you have to provide context, me...": If your project is open source already, it’s nice to know AI might be remembering the conversation. 14:37:55 From ts to Everyone: Great talk as always, Jacob! 14:38:09 From dteubl to Everyone: Good Talk, Great Summary! 14:38:12 From Eric Habets to Everyone: Thank you Jacob! 14:38:19 From Serge Malouin - IRAP to Everyone: I do not see the slides, will they be made avialable? 14:38:36 From srikanth shapur to Everyone: Thank You! 14:38:40 From Immanuel to Everyone: Thank you, Jacob. Good Talk! 14:39:35 From Harshad to Everyone: Does West replace the repo tool ? 14:40:21 From Viktor to Everyone: Thanks Jacob! How long did it take for you to master these 7 steps? 14:41:31 From Eric Habets to Everyone: Jacob, do you have an idea what the actual market share of Zephyr is? 14:42:00 From James Grenning to Everyone: Replying to "For me ceedling on C based, GoogleTest or CppUTest...": Be careful with ceedling... good tool, built by people that know what they are doing. Unfortunately ceedling can lead to over dependency on Mocks. Making tests more brittle and harder to read. 14:42:22 From Patrick Wright to Everyone: Replying to "Does West replace the repo tool ? ": Is it kind of like a fancy version of git submodules? 14:42:49 From ryan torvik to Everyone: “Letting git get out of control” - understatement of the day 14:43:10 From Jacob (Not Beningo :D) to Everyone: Replying to "Does West replace the repo tool ? ": I am not familiar with "the repo tool" (just using submodules here)... Is that https://umatechnology.org/guide-on-using-the-repo-tool/ this? 14:43:14 From Harshad to Everyone: Thank you Jacob, it makes sense! 14:43:23 From James Grenning to Everyone: Replying to "Anyone here using GoogleTest? Or what testing fram...": I use CppUTest. I am one of the authors. Build for embedded. Good for C and C++ 14:44:14 From Jacob (Not Beningo :D) to Everyone: Replying to "Anyone here using GoogleTest? Or what testing fram...": I am using Catch2 but might now migrate to CppUTest now that I know JG is on it :D 14:44:27 From James Grenning to Everyone: Replying to "Anyone here using GoogleTest? Or what testing fram...": CppUTest hides most the C++ syntax makeing it friendlier to C programmers. 14:45:03 From Harshad to Everyone: How hard is it to create such code reviewing interns? It's so cool if we can leverage something like this 14:45:06 From James Grenning to Everyone: Replying to "Anyone here using GoogleTest? Or what testing fram...": I have not used Catch2, but understand it is agoood tool. 14:45:37 From Serge Malouin - IRAP to Everyone: What CI tools are used? (I am still using a Jenkins building with batch file upon commits) 14:45:55 From Jacob (Not Beningo :D) to Everyone: Replying to "Anyone here using GoogleTest? Or what testing fram...": What I liked about Catch2 was how easy it was to set up with CMake https://github.com/catchorg/Catch2/blob/devel/docs/cmake-integration.md 14:48:06 From Otzen to Everyone: Replying to "When you integrate, the goal is to only find integ...": I like that formulation, going to one of my quotes :-) 14:48:06 From Jacob (Not Beningo :D) to Everyone: Replying to "What CI tools are used? (I am still using a Jenkin...": GitHub Actions here -- For STM32 there is a nice premade action in the marketplace that sets up the STM32CubeIDE (headless) build environment 14:50:56 From dteubl to Everyone: A great question from Tim! 14:51:21 From Otzen to Everyone: I have a sign on the wall in my office that says "Everybody has a testing environment. Some people are lucky enough enough to have a totally separate environment to run production in" 14:52:16 From ryan torvik to Everyone: Replying to "How hard is it to create such code reviewing inter...": Take a look at RooCode, Cline, and Claude Desktop. You don’t even need to write a handler, you can prompt the AIs to do ALL SORTS of things. 14:53:44 From Otzen to Everyone: My biggest chalenge in avoiding the mud ball, is that to everyone in management the mudball is invisible. They can't see what I see when I look at the code. If the physical production floor looked like that, there would bee a lot of big words frm management, because they can see it. 14:54:42 From Patrick Wright to Everyone: Replying to "My biggest chalenge in avoiding the mud ball, is t...": Maybe we can ask AI to "generate a picture representing this code" that makes it visible. 14:55:26 From ryan torvik to Everyone: Humans are so fascinating.... 14:57:31 From Otzen to Everyone: Replying to "My biggest chalenge in avoiding the mud ball, is t...": @Patrick Wright Yes, but can we convince the suits that this is a true picture? 14:58:23 From Jacob (Not Beningo :D) to Everyone: You mentioned Docker for build environment portability, but if I recall correctly, there are some restrictions for commercial use (requires paid license?). Are there other similar tools that are fully open-source / available for commercial use for free? 14:59:16 From Raul Pando to Everyone: Replying to "You mentioned Docker for build environment portabi...": Checkout podman 14:59:16 From Patrick Wright to Everyone: Replying to "My biggest chalenge in avoiding the mud ball, is t...": Since most of those people are saying "AI is the latest and greatest thing everyone should be using now. Have you released a product with AI yet?" Then I would hope so... 14:59:49 From Viktor to Everyone: Replying to "You mentioned Docker for build environment portabi...": Docker desktop for Windows requires one. But you can use a Linux CLI version. In Windows environment it can be run in WSL2. 15:00:35 From Jacob (Not Beningo :D) to Everyone: Replying to "You mentioned Docker for build environment portabi...": @Viktor That sounds about right. Unfortunately, that would be a big barrier to adoption for several on my team. 15:00:37 From Otzen to Everyone: Replying to "Humans are so fascinating....": That's right. Actually to make a well functioning architecture, it is 30% technical insight, and 70% insight in human (developer) behaviour. 15:01:28 From Harshad to Everyone: How about enterprise subscription of LLMs? I think that data is still private and not exposed publicly 15:03:08 From Otzen to Everyone: Replying to "How about enterprise subscription of LLMs? I think...": That is at least what is stated in the contract ;-) 15:05:19 From Otzen to Everyone: I have so far not found a free tool for adding unit-test on legacy/old code. There is a few comercial ones. 15:05:26 From Harshad to Everyone: Looking forward to the workshop on Friday. Thank you for a great talk 🙌 15:05:30 From Jacob (Not Beningo :D) to Everyone: 👏👏👏👏👏 15:05:51 From joao.junior to Everyone: 👏 15:06:57 From Immanuel to Everyone: Interesting Q & A too! 15:07:01 From Otzen to Everyone: @Jacob Beningo Only problem with this talk is that the topic is so big we could spend the entire conference discussing this 👍👍👍👍👍👍 15:07:17 From Lyden Smith to Everyone: Thanks Jacob! 15:07:30 From Eric Habets to Everyone: Thank you Jacob! 15:07:46 From kortej1 to Everyone: Thanks. Good presentation. 15:09:39 From Otzen to Everyone: Legacy code with tons of global variables, and pointers. Very hard to keep track on manually 15:12:23 From Otzen to Everyone: Replying to "Legacy code with tons of global variables, and poi...": I tried a comercial tool on one of our old functions, the function had the form "void foo( int var);" the tool exposed that there was 47 hidden in/out parameters hidden as global variables 😱 15:13:44 From Otzen to Everyone: Any ideas on visualizin the mudball to management? 15:14:15 From Serge Malouin - IRAP to Everyone: Replying to "Any ideas on visualizin the mudball to management?": Good ol Doxygen? 15:14:14 From Eric Habets to Everyone: Jacob, do you have an idea what the actual market share of Zephyr is? 15:15:10 From Eric Habets to Everyone: Zephyr is often mentioned in Job offers. 15:15:39 From Otzen to Everyone: Replying to "Any ideas on visualizin the mudball to management?": @Serge Malouin - IRAP I actually made a python script that would take the output from doxygen (xml) and create a dependeny map in dot (graphwiz) 15:16:06 From Eric Habets to Everyone: Ok, thanks!
Great talk, Jacob, thanks! I can't believe I'm the first to comment. Lots of good points. I think every project I've worked on suffers from those same legacy issues. I've tried introducing basic CI/CD, but often find it a hard sell, when people are so used to the legacy ways of working.
One thing I've often thought about, is whether code generation directly from a model could improve development practices and introduce fewer bugs. It's similar to how you recommend leveraging AI, but the model would be something the team maintain and is reviewable. Many years ago I used SCADE Suite in an aerospace job, the selling point being the generated code was qualified and so we only needed to work on and review the model for correctness. The code would just work and was always consistent (it was ugly and a pain to debug though). The generator template files could be made compliant with whatever coding standard and pass SAST. CubeMX is good with creating and changing skeleton code, but still needs manual code to work. Are there any tools you recommend, that preferably don't need a huge investment in time and money?