In practical GDB debugging experience, multithreaded debugging is more common, as many system designs are based on threads. Therefore, when a binary has issues, they often reside in a specific thread. At this point, GDB needs to correctly switch to the corresponding thread and then run.

If you simply switch to a specific thread and debug, this can actually be achieved according to previous articles. However, generally speaking, special handling is required for a specific thread, such as pausing the thread, observing a specific thread, or monitoring variables. This article mainly introduces these techniques.

Example Code

To illustrate these techniques, we need to prepare a multithreaded example program as follows:

#include <stdio.h>
#include <unistd.h>
#include <pthread.h>
int a = 0;
int b = 0;
void *thread1_func(void *p_arg) 
{
    while (1) {
        a++;
        sleep(1);
    }
}

void *thread2_func(void *p_arg) 
{
    while (1) {
        b++;
        a=b;
        sleep(1);
    }
}

int main(void)
{
        pthread_t t1, t2;
        pthread_create(&t1, NULL, thread1_func, "Thread 1");
        pthread_create(&t2, NULL, thread2_func, "Thread 2");
        pthread_join(t1, NULL);
        pthread_join(t2, NULL);
}

Just compile it:

gcc threads.c -lpthread -g -o threads

Let the Target Thread Run, Stop Other Threads

During debugging, we can focus on the thread of interest, stopping all other running threads to eliminate unrelated factors and locate the code. The main method is as follows:

set scheduler-locking on
b 9

At this point, based on the breakpoint at line 9<span>a++</span>, when the breakpoint is triggered, the value of b remains constant at 1. As shown below:

(gdb) p b
$1 = 1

As we can see, in this case, the two threads of the code do not run simultaneously when continuing; instead, thread2’s execution is stopped, and only thread1 runs.

In this situation, if we suspect that certain code is causing issues in a specific thread, we can use this method to eliminate possibilities and quickly determine the scope of the problem.

Stop the Target Thread, Let Other Threads Run

Unlike allowing the target thread to run, if we only want to debug a specific thread without stopping all threads, we can let other threads run while stopping the thread of interest to debug and check variables. As follows:

(gdb) set non-stop on
(gdb) r
(gdb) set scheduler-locking off
(gdb) b threads.c:16

At this point, we can check the thread status and see that thread1 is running while the breakpoint in thread2 is stopped, as shown below:

(gdb) info threads
  Id   Target Id                                     Frame
* 1    Thread 0xfffff7ff7010 (LWP 3262267) "threads" __pthread_clockjoin_ex (threadid=281474840293856, thread_return=0x0, clockid=0, abstime=<optimized out>, block=<optimized out>) at pthread_join_common.c:145
  2    Thread 0xfffff7de71e0 (LWP 3262274) "threads" (running)
  3    Thread 0xfffff75e61e0 (LWP 3262275) "threads" thread2_func (p_arg=0xaaaaaaaaaaf0) at threads.c:17

Thus, we can see that the value of b remains unchanged while the value of a keeps updating, as shown below:

(gdb) p b
$1 = 19
(gdb) p a
$2 = 96
(gdb) p a
$3 = 99
(gdb) p b
$4 = 19

Setting Watchpoints

During debugging, it is often necessary to set watchpoints for a variable within its scope. Generally, watchpoints can slow down the overall program execution, or even cause the code to run incorrectly, so hardware watchpoints (awatch and rwatch) should be used. Read/write watchpoints and read watchpoints need to be judged based on the business context, with read watchpoints typically having slightly better performance.

However, when using watchpoints, it is also important to pay attention to the scope; watchpoints that go out of the variable’s scope will become ineffective. Therefore, watchpoints for local variables need to be discussed separately; here we will first discuss watchpoints for global variables.

In multithreading, global variables are shared data. In this case, watchpoints will be effective in every thread. However, if I only want to observe a global variable in a specific thread, I can set the watchpoint to be effective only in that thread.

(gdb) awatch a thread 3
Hardware access (read/write) watchpoint 2: a
(gdb) c
Continuing.
[Switching to Thread 0xfffff75e61e0 (LWP 3349970)]

Thread 3"threads" hit Hardware access (read/write) watchpoint 2: a

Value = 10
(gdb) c
Continuing.

Thread 3"threads" hit Hardware access (read/write) watchpoint 2: a

Value = 11

As we can see, if thread 3 is specified, the read/write watchpoint for a will only be triggered in thread 3. This effectively sets a watchpoint that is only valid for a specific thread, making it easier to troubleshoot issues.

Conclusion

Many large programs are multithreaded. When performance issues or crashes occur, GDB is needed for troubleshooting. However, debugging multithreaded applications with GDB requires some techniques; otherwise, locating issues can be quite complex. This article introduces methods to improve the efficiency of multithreaded debugging, allowing GDB to directly identify problems without wasting time.