Debug approaches and technology development

Although embedded software developers would like to think that they spend the bulk of their time designing software and coding it, that is not the case. Almost everyone actually spends more time getting the code to function correctly - a process that we call "debugging". Different people have different ideas about what debugging actually entails and, with the advent of multicore designs, new challenges are presented. This article reviews debug approaches and technology and considers what is likely in the future.

What do you do?
If you met someone in a bar, say, and you got talking, they are likely to ask what you do. You will probably reply that you are an embedded software developer. If they carry on talking with you, they might ask you what that job entails and you would probably answer with something about designing software and writing C/C++ code. Of course, that is not really the case. Most embedded software developers spend the bulk of their time debugging.

This does beg the question: what does debugging actually mean? The answer is less simple than you might expect.


Debugging by hand: debug in the past

Many years ago, when computers were big and expensive, there were no debugging tools that we would recognize today. Frankly, programmers made fewer errors. One reason for this was that time on the computer was precious, so the programmer would write the code down on paper first, thinking it through very thoroughly. They would very carefully proof read to avoid syntax errors, as just one slip could waste a computer run. Then they would think through the operation of the code, dry running it on paper to check the logic. Eventually, this carefully scrutinized code would be submitted to the computer. After the run, the results would be studied and, if an error was detected, the code would be modified to include extra printf() (or equivalent) lines to show how values were changing. By using conditional compilation, "debug" and "production" versions of code could be compiled.

So, why not take this approach when writing embedded software today? The first part - writing the code on paper and thinking a lot - might be a very good practice. However, the ready access to computers that cost almost nothing has moved us away from such an approach. The practice of adding printf() calls to log values is still amazingly common, despite it being quite problematic for embedded applications. 

There are five key issues:

1. Most implementations of printf() are quite large. The function needs to address a very wide range of formatting situations and that takes a lot of code. This does not matter in a desktop software context, but memory is rarely in abundance in an embedded system.

2. There is the question of where the output of printf() actually goes. Many embedded systems have nothing that looks like a "console" and directing output back to a host computer can be challenging.

3. In a multi-threading context - i.e. when using an RTOS - reentrancy is a concern. If two threads try to use printf() at the same time, confusion can result.

4. Every time you want to change the variables you are looking at, a complete rebuild and download of the code is required.

5. Lastly, of course, formatting and sending the output (somewhere) takes time. If it is a real time application, this overhead can introduce problems.

A real debugger: Debug today

So, for most systems, using printf() as a debug tool is not viable and the use of a “real” debugger makes more sense. Debuggers are available from a wide variety of suppliers of embedded software development tools and can take a number of different forms, largely characterized by how/where the code gets to execute:

Simulation – It can be useful to start debugging before target hardware is available. One approach to achieving that is to use some form of simulation. This term covers a broad spectrum of technologies, which may include native execution of code on the host computer and instruction set simulation.

Virtual prototype – A step on from simulation is the use of a virtual prototype. A number of vendors offer tools whereby code may be executed and interact with a detailed simulation of the real target hardware at a number of possible levels of abstraction.

Network connection – If a target system is networked, this may be a good way to interface a debugger. However, it does demand fully working target hardware, functional network driver software and some kind of target “debug agent” software.

JTAG – Many embedded CPUs support the provision of a JTAG port and its inclusion has a minimal impact on system design. Debuggers that support a JTAG connection are widely available.

For many developers, the simplest solution is the last one listed – attach a debugger to a target system or an evaluation board using JTAG. This can give ready access to all the target data and addresses the five problems with printf()debugging thus:

1. There is no extra code on the target.

2. Data is displayed on the host computer without effort.

3. The debugger needs to be task-aware, but that is well understood.

4. Code only needs to be rebuilt to fix bugs.

5. A JTAG connection is slightly intrusive in the time domain, but the impact is typically quite small.

Debugging in the future – multicore and beyond

Debugging has always been challenging, as the manifestation of bugs can be very subtle. This became harder with the widespread use of multi-threaded code, but debugger suppliers responded well to the needs of developers. Many tools allow task-specific breakpoints and feature the option to stop the entire system or just the task of interest. However, this “stop and stare” approach becomes almost non-viable with a multicore system, which may also feature multiple operating systems and CPU architectures.

Going forward, it is likely that sophisticated tracing and analysis tools (such as Sourcery Analyzer from Mentor Graphics) will address the needs of developers of multicore systems, particularly with AMP (Asymmetric Multi-Processing) architectures. Although a tiny amount of code instrumentation is required, this has minimal impact. The large volumes of logged data can be analyzed to yield a clear picture of a system’s state and behavior in the lead up to an observed bug.

Baggio WANG FAN

SHENZHEN JAAPSON TECHNOLOGY CO LTD

baggio@jaapson-pcb.com  

www.jaapsonpcb.com

skype: baggiowang0214

JAAPSON, Expert in HDI Multi-layer PCB Manufacturing