With good debugging information, using GDB for debugging becomes very simple. This article introduces common debugging techniques that are frequently used in operating systems.

Compilation Environment and Runtime Environment

Before discussing common debugging techniques, we must clarify what a compilation environment is and what a runtime environment is. Essentially, when operating systems are released, the compilation environment is not the same as the runtime environment. The operating system distributor ensures that the basic libraries of the compilation environment and the runtime environment are as consistent as possible.

However, for GDB debuggers, there is a common issue here: the paths of the software’s compilation environment and runtime environment deployed on a certain embedded operating system are inconsistent. In this case, even if the binary contains DWARF debugging information, it is impossible to accurately view the line numbers. Below is an example to illustrate this.

Example Code

To demonstrate this issue, we need to prepare the example_print.c file with the following content:

#include <stdio.h>

int func1(int a)
{
    int array[100];
    int i;

    for (i = 0; i < 100; i++)
        array[i] = i;

    return array[99] * a;
}

int func2(int a)
{
    int b = 2;
    return b * func1(a);
}

int func3(int a)
{
    int b = 3;
    return b * func2(a);
}

int main(void)
{
    printf("%d\n", func3(10));
    return 0;
}

We will compile it with DWARF debugging information as follows:

gcc example_print.c -g -o print

At this point, we will place it in the runtime environment and debug it with GDB as follows:

scp print kylin@91:~

When loading with GDB, we can see that the DWARF line info is invalid.

# gdb ./print
Reading symbols from ./print...
(gdb) l
14      example_print.c: No such file or directory.

Handling Line Number Issues

In the above case, we need to obtain the corresponding source code as follows:

scp example_print.c kylin@91:~

Then we can load it using the directory command:

(gdb) directory ~/.
(gdb) l
14      int func2(int a)
15      {
16        int b = 2;
17        return b * func1(a);
18      }

At this point, we have resolved the common issue of missing source line numbers in operating systems.

Basic Debugging

Once we have the corresponding source line number, we can perform some basic debugging. Below are some very common debugging methods.

Printing Array Values

In the above code, the array array will be assigned values by i. If we want to analyze a specific value, for example, <span>array[55]</span>, we can do it as follows:

(gdb) u 11
(gdb) p array[55]@1
$4 = {55}

As we can see, this allows us to easily find the desired content in memory.

Printing Local Variables in the Stack

In the above code, each function has local variables. We can directly print the local variables of the upper stack in the bottom stack as follows:

(gdb) frame
#0  func1 (a=10) at example_print.c:11
11        return array[99] * a;
(gdb) bt
#0  func1 (a=10) at example_print.c:11
#1  0x00000055555508f0 in func2 (a=10) at example_print.c:17
#2  0x0000005555550920 in func3 (a=10) at example_print.c:23
#3  0x0000005555550944 in main () at example_print.c:28
(gdb) p func2::b
$5 = 2
(gdb) p func3::b
$6 = 3
(gdb) p func1::array
$7 = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99}
(gdb) p func1::i
$8 = 100

We can see that the code is currently on the stack of func1, but GDB can help us calculate all the local variables in the stack, and we can print them directly.

Manually Calling Functions

When debugging large codebases, some functions can be manually called, so we don’t have to rerun GDB. For example:

(gdb) set var a=100
(gdb) frame
#0  func1 (a=100) at example_print.c:11
11        return array[99] * a;
(gdb) call func1
$14 = {int (int)} 0x5555550844 <func1>
(gdb) c
Continuing.
59400
[Inferior 1 (process 3531) exited normally]

As we can see, after modifying the value of a, even though I am currently in func1, I can still actively call func1 again, thus multiplying the return value of func1 by 10.

Using C’s Jump Instruction

We can use C’s setjmp to implement code jumps, the most common being the goto statement. Similarly, GDB also provides the jump command. When debugging, if we want to return to a previous statement, we can directly use the jump command, as demonstrated below.

First, set a breakpoint at func2:

(gdb) b func2
Breakpoint 2 at 0x55555508e0: file example_print.c, line 16.

At this point, the code is at b=2, and the value of b is a random value from the stack, as follows:

(gdb) p b
$21 = 85

Assuming that during debugging, I pressed n, the code would reach the return statement as follows:

(gdb) n
17        return b * func1(a);

At this point, the value of b is 2, as follows:

(gdb) p b
$22 = 2

Now, using the jump command, we can return to line 16:

(gdb) jump 16
Continuing at 0x55555508e0.

Breakpoint 2, func2 (a=10) at example_print.c:16
16        int b = 2;

Since the jump command is consistent with C’s setjmp, the value of b is still 2:

(gdb) p b
$20 = 2

However, the code has now modified the PC register to return to line 16.

This method is very convenient for recording values that you forgot to log within a function, making debugging easier.

Conclusion

This article summarizes common techniques in GDB. First, it addresses the issue of missing source code in all systems, completing the DWARF information. Once the debugging information is complete, debugging can begin easily. It also introduces several small techniques in GDB to facilitate repeated debugging without forgetting to record key information, thus avoiding starting over in GDB.