Building a Simple Command-Line Interface
Why does Nathan recommend using HAL_UART_Receive_IT on the STM32 when building the CLI instead of a blocking UART receive call?
Hi Nathan. Great talk and examples! I am the same way - one of the first things I ever get working a serial port cli to easily test functionality and manually control or monitor things during operation, adding to the known commands as I go. I most always use the USART Rx ISR to store characters in a ring buffer, doing very basic parsing - ignoring white space, checking for sync characters, and ending a "packet" whenever a new line or whatever other END character I've designated comes through, and manually inserting a \0 into the buffer after a command packet. Then I set a flag to tell main that there is something available to read and it can deal with it when it has the chance.
Rather than use strings (because it's really inefficient in small MCUs and just storing the string commands in memory to reference uses a lot of space) I usually have very simple single character based command maps (L for LED, M for motor, e for enable, d for disable, etc) although as you had functionality that can get complex. So a command might look like L1e to enable LED 1. But then you can use a switch or even function pointer array to check for specific characters and launch an appropriate command handler function to deal with the rest of that packet. Looking at one character at a time makes it very easy and fast to parse info or numbers.
Transmitted responses from the MCU are also buffered, using the Tx ISR to send characters back out so main is never blocked waiting for slow comms.
One program I've found really helpful is Termite - it's a free RS232 emulator with helpful features like auto adding \r\n characters before sending and hex viewers to see raw data. It can handle RS232 handshaking, but I usually just use it with cheap USB to serial modules going straight to the USART Tx and Rx lines.
Thank you for your service, Nathan, and for the very well-done presentation. I'm sure this will be very helpful for people who are developing their first CLI.
One minor thing I didn't hear mentioned is checking for buffer overflow as each new character is received prior to adding it to the array. I'm sure you are aware, but less experienced users might just assume a newline will be received when expected, and if you make the array big enough, you'll be fine. However, buffer overflows are a common bug in embedded systems so one should always be aware of foreseeable misuse situations like my cat Skittles stepping on the keyboard while my CLI is running ;)
Yes, absolutely! In the Arduino code (which is C++), we're able to just continually add to the String variable (1) but in the STM32 code it's important to check for that. The sample code stops copying characters into the array if one of the terminating characters is received (\n or \r) or if the array fills up (see HAL_UART_RxCpltCallback()). In either case, the command is processed (even if it's malformed) and then the array is reset to receive another command over UART. Thanks for bringing that to everyone's attention and thanks for the wonderful compliment!
(1) Minor quibble: To be truly safe, I think even the String concatenation should be wrapped in try...except since it can still "overflow" if the system is unable to (re)allocate memory to store the now-larger string, but I'm not a really strong C++ programmer.
Nice talk Nathan, I like practical sessions like these!
The first thing that comes to mind with a CLI application interface is why not doing all the input/string pre-processing on the host side so as to relieve the microcontroller as much as possible. This way we could just implement a simple lookup table or hash table to trigger certain functions on the spot. What do you think?
Thank you so much! You're absolutely correct that compressing the messages in that way would result in shorter transmissions and (possibly) less processing on the MCU side. It's my understanding that webpages are infamous for having short, cryptic variable and function names for exactly that reason. I think this would only really help with the command lookup, though, or maybe parameters for which there is a fixed and small number of allowable options. Any other command line parameters (ones that allow the full range of integer or floating-point values or a user-defined string, for instance) couldn't really be substituted by a look-up. The other downside is the client-side processing, like you mention. I wouldn't be able to use a simple terminal program like PuTTY in that scenario, I'd have to have some kind of custom host-side program that could convert my user input into the command value that the MCU was expecting. But if the system needs the shortest transmissions possible, then it's definitely a design solution!
Really awesome talk. It was comprehensive and covered the all the points. It made me remember why got hooked with embedded in the first place. Thanks for sharing your experience and those extra resources. Those were really useful.
And that pyQt graph was really cool. I was really surprised to see that smooth updates without having to configure blitting. Last time I visualized with a graph with pyQt I was forced to discover blitting and then eventually use vispy to get the update rates I was looking for.
Thanks for the wonderful feedback! And I was also pleasantly surprised by the smoothness of the graph. I've had issues with update rates in the past (like you) and I'm wondering now if it doesn't have something to do with the nature in which the graph gets updated. In the version I demonstrate, there's a timer function that runs every 10ms which sends "r" to the MCU and then just converts the response to int and adds it to the graph, so it's basically polling the MCU. A previous version of that demo had the MCU sending data at a specified rate and when the timer function ran it would read serial characters from the buffer that's on my laptop until there were no more characters to be read. Despite the data being sent at the same rate (100Hz), that version would freeze and I'm wondering if it might be a result of the timer function running longer than 10ms and getting triggered again before it has finished...
Really worth to listen, Nathan. Espechially thanks for the getline link!
Thank so much! It's really a blessing to be able to share something with others that they find so useful.
hey Nathan, thank you for your service and for the great presentation. It made me want to explore more of the simple CLI interfaces for my future projects... you never know when you'll have to fix / adjust / monitor something after the project is complete!
True! I can use a debugger about as well as the next person but for many projects I find that a CLI ends up being a better fit (for fixing/adjusting/monitoring, like you say). And thanks so much for the compliment!
Thanks for the talk. Great example and demonstration on why a CLI is important and how to build one. One great example of CLI interface I have used recently is the Zephyr Shell. It handles lots of special ASCII characters as well as autocomplete and color coding.
Very cool! And thanks for the compliment. I also noticed that the Zephyr shell had some neat features when I was watching Bejamin Cabe's talk. I wonder how large the Zephyr code gets to use all of that, though.
I enjoyed this talk, thank you. I think security would be important in many applications. In a real product it might be useful to leave the CLI in the production code, so that it can be used afterward on products in the field, or maybe it could be used by manufacturing for run-time configuration, etc. But it would be good to be able to secure such things, and I think a simple "password" probably wouldn't cut it - at least not in many scenarios. So that's something to look into further for sure. In any case, even just as a debug aid that doesn't get left in the production code, a CLI as you've described is very useful.
Great point! I made a mention of that at the end but since security isn't my area of expertise I didn't have much to offer. If you have any good references for securing a CLI, I'm sure the folks here would be happy to hear it.
The CLI is dead, long live the CLI :-)
I'm doing embedded development for nearly 17 years now and would be a rich man if I got 1$ every time CLI was told dead. In fact a CLI is, especially during developent and testing, a really cheap and useful support tool. And if secured enough also for products in the field.
Agreed! Especially if you consider it a "simplest possible version" of just "MCU to computer" communication. The channel could be I2C or USB or WiFi, the codes/commands send back and forth could be enumerated and have verification features like TCP/IP, etc, but the simplest thing to do is just send "on" in ASCII over UART.
Hi, Nathan,
This video is great. Do you have a link to download the source code of CLI. Thanks
I do! It currently lives here.
Thanks for this talk, Nathan! I've often thought this kind of thing would be a good tool and I suppose most of us end up making some variant as we get used to a new dev board or application. On the PC side you demo'd a serial connection and a custom Qt program. ST have their monitor which can read serial interfaces and do some processing, graphs, tables etc. (https://www.st.com/en/development-tools/stm32cubemonitor.html) Have you, or anyone else, come across similar tools that are open source or do a better job of this?
I have used Serial Studio a couple of times and found it great for quick visualization of data sent over UART (or other methods) : https://github.com/Serial-Studio/Serial-Studio
As long as you do non-comercial development SEGGER SystemView is something worth taking a look at (also as a company the benefit of having such a tool is always worth the price). And afaik Quantum Leaps offer their tools for free as long as GPL licensing is OK for you.
I don't know of any open source options but it's my understanding that Freescale has a program called FreeMaster for their products and Micirum (bought by Silicon Labs; can't find a link rn) offers uC/Probe, both of which do the same thing. uC/Probe has the advantage of not being restricted for use with only a single vendor's microcontrollers (unlike FreeMaster or STM32Monitor). For simply monitoring and/or adjusting system variables on a microcontroller, you're right, those programs probably offer much more capability than a homegrown GUI + CLI, like I demonstrated. The biggest downside, though, is that I wouldn't consider them "beginner friendly"; I would think you would need at least an intermediate knowledge of the build process and of the OS/scheduling code in your application to integrate those applications successfully.
Hi Nathan, I enjoyed your talk very much, as it was so refreshing with the hand-drawn flow diagrams. We had a similar talk on last years EOC. A CLI provides great value and insights for development. I also liked the different platforms you addressed (Arduino and STM). Thank you again!
Thanks so much! It's my only desire to share something useful with people so it means a lot that you'd take the time to leave a comment. :)
I also love the hand drawn diagrams but I'm more curious as to what you're using for that. They seem to be part of some system that supports whiteboarding so I'm always curious about "live" options for that. Is it anything special?
Nothing special, I think! It's just Microsoft OneNote with a Wacom for pen capture. I started using this setup to write out the notes I prepare for the classes I teach and I like it a lot, as compared to Word/Powerpoint.
Very good presentation. Even though I already have used CLI in my projects I still find this useful and informative.