Home > On-Demand Archives > Q&A Sessions >
Live Q&A - Tools and Techniques to Debug an Embedded Linux System
Sergio Prado - Watch Now - EOC 2022 - Duration: 20:54
Sergio, absolutely fantastic presentation! Thanks for sharing such an amazing content!
I've a question for you: to get kernel logs when we're developing a product, it's usual to configure a serial console for kernel and check the messages sent there (as I believe you've done when checking kernel crash in USB sub-system) during developing and testing phases. However, sometimes we're in a situation where there isn't a "free" uart for this (like debugging a product which a bug has been discovered in field operation, for example). In these cases, what's the best way to get kernel crash logs? Could you please share some good practices in this situation?
Thanks in advance.
Hi phfbertoleti! I am glad you enjoyed it!
About your question, you have a few options:
- You could redirect kernel logs (including kernel oops messages) to some persistent storage that you could retrieve later. Usually, log daemons like journald or rsyslogd are able to do that.
- You could use the kernel pstore framework. This framework enables you to store kernel logs (including crashes) in a separated region of memory that you could retrieve later (after a soft reset for example).
Hope that helps.
Thanks!
This was a fantastic talk, you are spot on debugging is not given anywhere near enough attention which it certainly deserves! I also loved "THE SIX STAGES OF DEBUGGING", classic!
I am glad you liked it!
Will the video be available for download? You covered a lot more information during the debugging exercises than are covered in the slides.
If you mean the video of the presentation, they cannot be downloaded. But they can be watched at will on the website.
Awesome session, extremely informative!
Thanks @jgaitan! I am glad you enjoyed it!
10:44:54 From Raul Pando : Hi Sergio, great presentation. Most of the techniques you presented seem to require to a binary compiled with debug symbols enabled. I was wondering if any of them can still be used with a release binary that doesn't have those symbols 10:47:04 From Raul Pando : Thanks! 10:48:37 From Paul Vittorino : What about embedded environments without Ethernet? What are the options for GDB remote? 10:51:02 From Tim Michals : Have you used jtag via openocd to debug the Linux kernel? 10:57:01 From Tobi : Thanks for the talk Sergio! Can you comment about log storing and reporting for post moretm analysis? In an application with limited flash storage, and considering flash wear and also cellular data usage, sometimes it is difficult to find the relevant lines of logs on a deployed device. 10:57:38 From Tobi : Specially when there are many units running, and logging to journalctl 11:00:32 From Michael Kirkhart : Yes, I seem to remember this was the cause of problems with earlier Tesla vehicles - the FLASH would wear out because of the volume of logging, and this essentially disabled the vehicle. 11:00:40 From Tobi : That really makes sense 11:00:44 From Tobi : Thanks Sergio 11:00:54 From Stephane : Thank you Sergio! 11:00:56 From Tim Michals : thank you.. 11:00:57 From Maxime L. : Thank you! 11:01:49 From Mathieu. P : Thx, very interesting presentation
Thank you very much Sergio, great presentation!