Home > On-Demand Archives > Talks >

Best Practices for RTOS Application Design

Jacob Beningo - Watch Now - EOC 2021 - Duration: 50:58

Real-time Operating Systems (RTOS) have been finding their way into nearly every embedded system both connected and disconnected. They provide a convenient paradigm for designing flexible and scalable systems, interacting with hardware, scheduling tasks and many other capabilities. The problem though is that RTOS best practices are not well documented and even seasoned developers can run into costly and time-consuming issues that could have been avoided.

In this session, Jacob Beningo will walk attendees through best practices for RTOS application design such as:

  • Decomposing an application into tasks
  • How to set task priorities
  • Analyzing applications data flow
  • Useful design patterns handling events, interrupts and initializing tasks
M↓ MARKDOWN HELP
italicssurround text with
*asterisks*
boldsurround text with
**two asterisks**
hyperlink
[hyperlink](https://example.com)
or just a bare URL
code
surround text with
`backticks`
strikethroughsurround text with
~~two tilde characters~~
quote
prefix with
>

Nam
Score: 0 | 3 years ago | 1 reply

Hi Jacob,
Thank you very much for your talk.
I saw you have too many books behind you. Could you suggest me some of good books you have for embedded engineers?

Jacob_BeningoSpeaker
Score: 0 | 3 years ago | no reply

Certainly!

Below are a few good ones and classics:

  • Clean Architecture by Robert Martin
  • Real-time Concepts for Embedded Systems by Qing Li
  • Test Driven Development for Embedded C by James Grenning
  • Real-time C++ by Chris Kormanyos
  • Definitive Guide to ARm Cortex-M23 and Cortex-M33 Processors by Joseph Yiu
  • The Art of Designing Embedded Systems by Jack Ganssle
  • C in a Nutshell by Peter Prinz and Tony Crawford
  • Expert C Programming Deep Secrets by Peter Van Dev Linden

Obviously there?s a ton more, but hopefully that helps give you some reading ideas!

aldo_coronado
Score: 0 | 4 years ago | 1 reply

What is the best way to implement an application which has an inherent need for state control
(i.e. state machine, controlled by driver inputs) in and RTOS design paradigm?
I have implemented an Event Handler which distributes messages from a Queue to the running context, but so far hasn't been deterministic.

Jacob_BeningoSpeaker
Score: 0 | 4 years ago | no reply

Typically I run the state machine within the task using a StateMachineRun function. I then expose API's that can be used to trigger state transitions from other context.

Serg
Score: 0 | 4 years ago | 1 reply

How to handle Asynchronous data reception in the system. For example if there 2 UART peripherals(no flow ctrl), that should accept data, that later should be processed by application. How should this be handled ?

Jacob_BeningoSpeaker
Score: 0 | 4 years ago | no reply

You could use a UART Gate Keeping task to manage it. You could also use a semaphore to signal which UART the data came from. Also a queue. Many different patterns you can use. let me know if that makes sense.

SimonSmith
Score: 0 | 4 years ago | 1 reply

Jacob, great talk and Q&A session. I?d love to use Tracealyzer to learn more about what my application is really doing under the hood, but it?s just too expensive for occasional use. Have you found others such as SystemView to be almost as useful? Also I?m using ST-Link v2 on a custom STM32 board for debugging, but the turnaround time after making a change and loading into flash takes a minute or so, multiple times during the day. I know ST-Link v3 exists, but there seem to be multiple models and doesn?t seem to be available. Is something like I-Jet or J-link really any faster in practice? I know they offer unlimited breakpoints, which would help. Are they also fully compatible with custom STM32 boards (i think using SWO)? Thanks!

Jacob_BeningoSpeaker
Score: 1 | 4 years ago | no reply

Thanks for the question.
SystemView can give you some details, but not as powerful as Tracealyzer. STM32CubeMonitor could also be helpful. Not used it much yet, but looked interesting.

I'm biased here, in that I always find paying for a tool is well worth the cost. So if you work for a business, I'd recommend you get them to just buy the Tracealyzer license. My technique is to convert the cost for a tool to the number of hours of my labor it takes to equal it. So if the tool is $1500 and I make $50/hr, then if the tool will annually save me at least 30 hours of work and debugging, it's well worth the investment.

I have not used the ST-Link v2 in quite some time. I use my J-Link Ultra+ most of the time. Typically the erase and load time is < 5 seconds. You may be able to update the ST-Link Firmware to V3.

JeanLabrosse
Score: 0 | 4 years ago | 2 replies

The FreeRTOS CPU usage function doesn?t really work.

SimonSmith
Score: 0 | 4 years ago | no reply

Its great to hear a few mentions of uC/OS-III. I use it every day and the book is very clear. I enjoyed your talk about MPUs also.

Jacob_BeningoSpeaker
Score: 0 | 4 years ago | no reply

Thanks! That is good to know!

daleonpz
Score: 0 | 4 years ago | 1 reply

Great talk Jacob, how do you perform a Worst Case Stack Analysis? by checking all the variables within the task and sum the memory spaces?

Jacob_BeningoSpeaker
Score: 0 | 4 years ago | no reply

There are two main ways I do it.
1) Analytical by using a static code analyzer. (Can also do it manually, but the code analyzer is faster)

2j through testing. Can use tools like RTOS aware debugging, ides, Tracealyzer, rtos stack monitors, stack watermarks, etc.

nathancharlesjones
Score: 0 | 4 years ago | 1 reply

Jacob, I actually have a question about your 2020 talk "Developing Reusable Firmware for MCUs", but this seemed the best place to post it. In that video, some of the peripheral APIs you showed had a "CallbackRegister" function for which the function pointer had the formate "TYPE (*CallbackFunction)(type)". What are "TYPE"/"type" in that function signature? Thanks!

Jacob_BeningoSpeaker
Score: 0 | 4 years ago | no reply

Thanks for the question. Yes so TYPE was my generic way of putting in some pseudo code that a developer would update. So TYPE would be replaced by uint32_t, uint8_t, or whatever type the application required. We could probably make it void or some generic return that is case into what is needed.

leandropg
Score: 0 | 4 years ago | 1 reply

Hello Jacob... I have some questions:

1) I have a task that handle the UART messages logger. I create a queue that all the other task use for send message to the UART logger. The problem is that the object of the QUEUE that store the message consume much memory, because for example save 256 bytes to the message. If I defined a queue of the 200 elements... the queue use 51.2KB from RAM... What you suggest me to reduce this high consumption?

2) Related with "Best Practices for Setting Task Priorities"... What is the difference in the shortest response time vs shortest execution time? I understand execution time, but I don't understand the concept of shortest response

3) How measure the response and execution time of a Task?

4) How I can document a RTOS project (a tool) that show the relation between task, priorities, sempahores and mutexes?

Jacob_BeningoSpeaker
Score: 0 | 4 years ago | 1 reply

Thanks for the questions. Please see below:

1) I would suggest that you pass the data by pointer. You can have a shared memory location of 256 bytes that stores several messages. Then pass the pointer to the memory location.

2) Basically the task that needs to respond quickly has the higher priority. For example, the response time on a vehicle brake peddle may be 30 ms, while the knob on a radio might be 100 ms. The brake peddle should therefore have the higher priority.

3) I use Tracealyzer to measure how my system is performing. It measures response time, execution time, periodicity and several other metrics.

4) I typically use a tool like Visio, eDraw Max or even sometimes PowerPoint.

leandropg
Score: 0 | 4 years ago | no reply

Thanks Jacob

billionai
Score: 0 | 4 years ago | 1 reply

Great talk, and very cool approach for a lot of embedded, not just RT stuff! I'll try and apply it more in my general life as well

Jacob_BeningoSpeaker
Score: 0 | 4 years ago | no reply

Thanks! I?m glad to hear that you like the approach! I hope it helps!

Eduardor
Score: 0 | 4 years ago | 1 reply

Concerning to setting task priorities, what is the difference between Response Time and Execution Time?

Jacob_BeningoSpeaker
Score: 0 | 4 years ago | 1 reply

Response time will look at how quickly the task needs to start executing when it becomes due to run. Execution time is just based on how much CPU time the task needs to runs.

Eduardor
Score: 0 | 4 years ago | 1 reply

I was not sure concerning to the meaning of response time in this context, so you've clarified it. Thanks! And congratulations for the presentation, it was excellent!

Jacob_BeningoSpeaker
Score: 0 | 4 years ago | no reply

Thanks!

nathancharlesjones
Score: 0 | 4 years ago | no reply

Also, regarding the summary slide: Does your book on reusable firmware talk about how to build a diagnostic/fault detection task? What are some other resources you might recommend to learn more about implementing that?

JeanLabrosse
Score: 0 | 4 years ago | 1 reply

I agree that generally you should do: Input-Process-Output but sometimes you may need to do Output-Input-Process especially in periodic tasks. The reason is to ensure that outputs always occur at predictable intervals. In you input and process, depending on the processing time for these then you would create jitter in the output is might not be acceptable. In fact, I would update ALL the outputs first and then read all the inputs and then do the processing afterwards.

Jacob_BeningoSpeaker
Score: 0 | 4 years ago | no reply

Great suggestion! Totally agree!

JeanLabrosse
Score: 1 | 4 years ago | 1 reply

One thing that would be worth mentioning is that all the RTOS services consume CPU cycles and thus the application developer needs to be mindful of that. So, yes, you should use RTOS services but be aware of the impact of performance.

Jacob_BeningoSpeaker
Score: 1 | 4 years ago | no reply

Yes great suggestion / observation! These are often easy to overlook.

JeanLabrosse
Score: 0 | 4 years ago | 1 reply

I like the idea of your task creation pattern. Makes it easier to make changes. However some off-the-shelf modules have their own initialization code so that cannot be called. Also, what do you do with other kernel objects?

Also, you used power-of-two stack size, any specific reason or, to make it easier to use with an MPU?

For task stack storage, you should never use dynamic memory location UNLESS you never delete tasks at run time which you should not anyway.

Jacob_BeningoSpeaker
Score: 1 | 4 years ago | no reply

Thanks. So, I try to use this pattern if I can to manage my application tasks when it makes sense. Certainly third party libraries will create their own tasks such as spinning off TCP/IP, MQTT or other tasks. To find these, I usually have to trace the system.

With the other kernel objects, I generally try to abstract them into the modules that use them so that they are hidden.

To make it easier to use with an MPU ... ;-) After doing this for 20+ years I generally do everything in powers of two so it's just natural. Moving averages need to be done, power of two elements. Allocate memory, power of two. etc.

I 100% agree on never using dynamic memory. Unfortunately my examples are lazy in that they use dynamic memory. Prior to FreeRTOS v8, they didn't even support static allocation so the only way to safely allocate a task was to dynamically allocate it at start-up and then leave it there.

MrPotter
Score: 0 | 4 years ago | 1 reply

Are there situations where there is a value in reducing the number of interrupts by configuring some slow peripherals for polling instead of interrupts, assuming your system can can service them fast enough at a fixed rate?

Jacob_BeningoSpeaker
Score: 1 | 4 years ago | no reply

Thanks for the question. There can be situations where it makes sense to avoid using the interrupts or at least need to be super careful how they are used. For example, if you are using input capture to calculate frequency or duty cycle and calculating in the ISR, you may find that with signals that are 5 KHz and up that most of your time is spent servicing the input capture interrupts. In these instances, it can make more sense to poll the timer at a known interval and take an average of the edges that were detected ( well at least for frequency measurement. That won't work for duty cycle).

garyhubley
Score: 0 | 4 years ago | 1 reply

Do you have a syntax or reference document for the data flow diagram you created? I have not seen that syntax before and I like it.

Jacob_BeningoSpeaker
Score: 1 | 4 years ago | no reply

Thanks for the question. I don't actually. It's just notation that I started to use when I started to use RTOS based systems. I will do some checking to see if there is anything already formalized and if not, maybe i'll look to do so. Thanks for the feedback/question.

Serg
Score: 0 | 4 years ago | 1 reply

How to handle Asynchronous data reception in the system. For example if there 2 UART peripherals(no flow ctrl), that should accept data, that later should be processed by application. How should this be handled ?

Jacob_BeningoSpeaker
Score: 0 | 4 years ago | no reply

There are different design patterns that can be used to handle something like this. First, I generally would have a single task to handle the incoming UART data. No need for two tasks to interface to similar devices. I would then use an interrupt, either from the UARTs or even DMA, to signal via a semaphore that there is data to be processed. The ISR would push the incoming byte into a circular buffer and then the UART task would manage the data that was received. This will vary from application to application but usually its served up to another function or task that validates it and figures out what to do with it.

RoccoBrandi
Score: 0 | 4 years ago | 1 reply

what is the difference between Response Time and Execution Time? what can influence the response time of a task?

Jacob_BeningoSpeaker
Score: 0 | 4 years ago | no reply

Thanks for the question. Execution time is how long it takes for the task to run. Response time is how long it takes until the task is able to run.

Javi
Score: 0 | 4 years ago | 1 reply

Hi Jacob, great talk. I want to ask you about low power techniques in RTOS. Apart from putting the MCU to sleep in the idle task, are there more things to consider? what approach do you usually use for battery powered systems where low power is crucial?

Jacob_BeningoSpeaker
Score: 0 | 4 years ago | no reply

Thanks for the question. For low power systems, I generally try to make everything as event driven as possible. I try to eliminate periodic tasks. An event wakes the system up and signals a task that needs to run, it runs signals any others and when done the idle task is there to put the system back to sleep.

OUR SPONSORS

OUR PARTNERS