Beyond breakpoints: Advanced debugging for RISC-V-based applications
Written by Rafael Taubinger, Global Product Marketing Manager, IAR
Debugging is always a large part of development efforts, and the task can be exhaustive and, in the worst case, ineffective. For example, an application that has been written to make use of a multitasking operating system and event-driven interrupts. It will make debug methods such as printf instrumentation or setting breakpoints and single-stepping the application tedious and unproductive, but even worse. Developers are also likely to miss the most complex timing issues as the core is constantly being halted.
A product's quality will only be as good as a developer's debugging capabilities. Developers need to be able to analyze and track the exact root of a specific bug or understand what is happening on every line of code. Without such capabilities, developers might mainly apply workarounds using best guesses instead of fixing the real issues.
Depending on the maturity of the development organization, developers can spend up to 80% of their time debugging. If they could isolate defects before they make it into a release build, they would have a lower defect injection rate. This would mean that the organization’s quality metrics could be reached much more quickly, and the development efforts overall could be reduced.
RISC-V core and debugging capabilities
The RISC-V Debug Support Specification [1] specifies the debug interface for RISC-V-based devices or SoCs. This article will not discuss the interface specification, which was ratified a few years ago, in detail.
The RISC-V External Debug interface, in combination with a capable debug probe like I-jet or similar probes [2], enables developers to examine the application’s behavior from various angles. Together with a toolchain, the debug architecture can make debugging quick, easy, and precise.
On top, there are also ways to analyze an application even more in detail through trace. The RISC-V International organization has been working to standardize trace specifications for RISC-V. Two main task groups have been working on the specifications. The first task group is the Processor Trace working group [3], which has the first ratified version of the standard describing the trace format, also known as E-Trace for RISC-V. The second is the RISC-V Nexus Trace group [4], which is working with recommendations on how to use trace as defined by the Nexus IEEE-ISTO 5001 standard for RISC-V cores.
Implementing trace IP in a device brings the possibility to trace the application as it is running non-intrusively. In a detailed approach, trace is a continuously accumulated sequence of each instruction executed for a selected part of the application. There is no speed/code penalty when using trace, so that the application will run smoothly during the process. However, there is a tradeoff in adding some cost to the final silicon with trace and probes tend to be more complex, but the benefits will pay off easily for embedded developers.
On the other hand, the good news is that high-end debugging probes that also support trace are now available for a relatively lower cost.
General features of a high-level-language debugger for RISC-V
A high-level-language debugger for embedded applications is designed for use with C/C++ compilers and assemblers, providing development and debugging within the same application. This gives more possibilities, such as making corrections directly in the same source code window used for controlling the debugging during a debug session, inspecting and modifying breakpoint definitions when the debugger is not running, and allowing breakpoint definitions to flow with the source code while editing.
The extensive debug information for the debugger generated by the build tools (ELF/DWARF output with debug symbols), combined with the RISC-V Debug Specification [1], results in good debugging possibilities shown in Picture 1.
.png "Picture1 (1)")
Picture 1. Example of a high-level-language debugger for RISC-V
With a closer look, it’s possible to observe that a high-level-language debugger allows to switch between source and disassembly debugging as required, for both C or C++ and assembler source code.
Additionally, compared to traditional debuggers, where the finest granularity for source-level stepping is line by line, a high-level-language debugger provides a finer level of control by identifying every statement and function call as a step point. This means that each function call inside expressions, and function calls that are part of parameter lists to other functions can be single-stepped. This is especially useful when debugging C++ code, where numerous extra function calls are made, for example, to object constructors.
Furthermore, the high-level-language debugger breakpoint system allows setting breakpoints of various kinds in the application being debugged, allowing it to stop at locations of particular interest. For example, set breakpoints to investigate whether the program logic is correct or to investigate how and when the data changes.
Likewise, for variables and expressions, there is a wide choice of features. It is possible to monitor the values of a specified set of variables and expressions continuously or on demand. It is also possible to choose to monitor only local variables, static variables, etc.
Moreover, when an application is executed in a high-level language debugger, the elements of library data types, such as STL lists and vectors, can be viewed. This gives a very good overview and debugging opportunities when working with C++ STL containers.
Again, the compiler generates extensive call stack information. This allows the debugger to show, without any runtime penalty, the complete stack of function calls wherever the program counter is. The debugger can select any function in the call stack and for each function to get valid information for local variables and available registers.
Finally, RTOS awareness gives a high level of control and visibility over an application built on top of an RTOS. It displays RTOS-specific items like task lists, queues, semaphores, mailboxes, and various RTOS system variables. Task-specific breakpoints and task-specific stepping make it easier to debug tasks.
Breakpoints and the essence of debugging
The fact is that developers cannot live without breakpoints. A high-level debugger allows setting various types of breakpoints in the application to be debugged, allowing stopping at locations of particular interest. It is possible to set a breakpoint at a code location to investigate whether the program logic is correct or to get trace printouts. In addition to code breakpoints, additional breakpoint types might be available. For example, it might be possible to set a data breakpoint to investigate how and when the data changes. Additionally, it lets the execution stop under certain conditions, which can be specified. The breakpoint can also trigger a side effect, for instance, executing a macro function, by transparently stopping the execution and then resuming. The macro function can be defined to perform a wide variety of actions, for instance, simulating hardware behavior. All these possibilities provide a flexible tool for investigating the status of the application.
Different types of breakpoints supported in a high-level debugger
The different types of breakpoints can be used to investigate different types of issues or even be combined for the best results. Let’s expand and exemplify the most common use cases.
The code breakpoints are used for code locations to investigate whether the program logic is correct or to get trace printouts. Code breakpoints are triggered when an instruction is fetched from the specified location. If the breakpoint is set on a specific machine instruction, the breakpoint will be triggered, and the execution will stop before the instruction is executed.
Also, log breakpoints provide a convenient way to add trace printouts without having to add any code to the application source code. Log breakpoints are triggered when an instruction is fetched from the specified location. If the breakpoint is set on a specific machine instruction, the execution will temporarily stop and print the specified message in the debug log window.
Picture 2 shows the use of breakpoints in a RISC-V high-level language debugger. Notice that all breakpoints can be disabled and kept in their original locations for later use.
.png "Picture2 (1)")
Picture 2. Example of breakpoint usage in a RISC-V high-level-language debugger
Additionally, the data breakpoints are primarily useful for variables that have a fixed address in memory. If a breakpoint is set on an accessible local variable, the breakpoint will be set on the corresponding memory location. The validity of this location is only guaranteed for small parts of the code. Data breakpoints are triggered when data is accessed at the specified location. The execution will usually stop directly after the instruction that accessed the data has been executed.
Further, data log breakpoints are triggered when a specified memory address is accessed. A log
entry is written in the data log window for each access. Using a single instruction, the microcontroller can only access values that are four bytes or less.
Furthermore, a high-level debugger also supports immediate breakpoints, which will halt instruction execution only temporarily. This allows a macro function to be called when the simulated processor is about to read data from a location or immediately after it has written data. Instruction execution will resume after the action. This type of breakpoint is useful for simulating memory-mapped devices of various kinds (for instance, serial ports and timers).
Finally, also worth mentioning the trace start trigger and trace stop trigger breakpoints to start and stop trace data collection, a convenient way to analyze instructions between two execution points. Trace breakpoints are only available for probes and devices supporting a RISC-V Trace implementation.
Monitoring stack memory usage
Handling the stack is one of the major challenges for embedded software developers. Proper stack configuration is essential to the system's stability and reliability. If the stack size is too small, an overflow situation could occur. On the other hand, setting the stack size too large means wasting RAM resources, which could be limited in some RISC-V-based embedded systems.
A professional development tool should make it possible to estimate via the compiler and linker and to control and monitor the stack usage via the debugger.
Picture 3 shows an example of a monitoring stack. When the application is first loaded, and upon each reset, the memory for the stack area is filled with the dedicated byte value like, for example, 0xCD before the application starts executing. Whenever execution stops, the stack memory is searched from the end of the stack until a byte whose value is not 0xCD is found, which is assumed to be how far the stack has been used. The light gray area of the stack bar represents the unused stack memory area, whereas the dark gray area of the bar represents the used stack memory. For this example, it is possible to see that only 44% of the reserved memory address range was used, which means that it could be worth considering decreasing the size of memory.
Picture 3. Example of monitoring stack memory usage
Although this is a reasonably reliable way to track stack usage, there is no guarantee that a stack overflow will be detected. For example, a stack can incorrectly grow outside its bounds and even modify memory outside the stack area without actually modifying any of the bytes near the end of the stack range. Likewise, the application might modify memory within the stack area by mistake.
How can Trace make a difference?
Trace is known to help quickly diagnose common problems that are nearly impossible to find without it. By using a trace, it is possible to inspect the program flow up to a specific state, such as an application crash, and use the trace data to locate the origin of the problem. Trace data can be useful for locating programming errors that have irregular symptoms and occur sporadically.
To use trace, the target system must generate trace data. Once generated, the high-level-language debugger can collect it via the trace probe [2], and developers can visualize and analyze the data in various windows and dialog boxes.
Picture 4 shows how to use the trace-related windows, including trace instructions, function trace, and a graphical timeline with all function calls and timing information.
When trace data is collected, it is possible to perform searches in the collected data to locate the parts of the code or data that might be interesting, for example, a specific interrupt or accesses of a specific variable. Additionally, the code coverage functionality available with the trace is useful when a test procedure is designed to verify whether all parts of the code have been executed. It also helps to identify parts of the code that are not reachable. This is highly valuable and mandatory by the safety standards when working with critical safety applications.
Picture 4. IDE and trace visualization from trace information provided from the device and in sync with the C/C++ code
The trace functionality is extremely valuable. As explored in detail in the article Trace Techniques for RISC-V—How to Use it Efficiently [5], trace could help you find “million-dollar” bugs in your application.
Get help from the right debugging tools.
Only those who have ever experienced a hard fault in an RISC-V-based design know how difficult and frustrating it can be to track down the ultimate issue. It is not uncommon to spend several days trying to isolate the issue and come up with a fix or end up with a poor workaround. Some bugs might be difficult to catch and only pop up randomly and in elaborate circumstances.
A high-level-language debugger with useful functionality like complex and conditional breakpoints, data and log breakpoints, macros, stack monitoring, and trace on top is a shortcut that can boost efficiency while gaining complete control of every line of code and every single instruction executed in the complex application. By using the right debugging tools (that all developers deserve) in the right way, it’s possible to shorten the development cycle by fixing bugs for real.
References
[1] https://github.com/riscv/riscv-debug-spec and https://riscv.org/wp-content/uploads/2019/03/riscv-debug-release.pdf
[2] https://github.com/riscvarchive/riscv-software-list
[3] https://github.com/riscv-non-isa/riscv-trace-spec
World leader of software and services for embedded systems development.
© 2024 IAR Systems