Introduction:Recently, while reviewing my notes, I found a solution introduced by a colleague in June regarding event collection for security products in Linux: kprobe. I have also made some understanding based on the source code and recorded it.Note: Compared to my knowledge accumulation in Windows, my understanding of Linux is relatively shallow. If there are any issues with my analysis, please feel free to correct me~Content:
From various learning materials, it can be seen that kprobe uses int 3 to hijack the execution of specified functions in the x86 architecture. However, the current x86_64 architecture supports a jump optimization scheme called Jump Optimization, which requires the kernel to enable the CONFIG_OPTPROBES option. This scheme uses jump instructions instead of breakpoint instructions, and the machine being analyzed is configured with this type of jump.
First, let’s take a look at the definition of the kprobe structure:
struct kprobe { struct hlist_node hlist; /* list of kprobes for multi-handler support */ struct list_head list; /*count the number of times this probe was temporarily disarmed */ unsigned long nmissed; /* location of the probe point */ kprobe_opcode_t *addr; /* Allow user to indicate symbol name of the probe point */ const char *symbol_name; /* Offset into the symbol */ unsigned int offset; /* Called before addr is executed. */ kprobe_pre_handler_t pre_handler; /* Called after addr is executed, unless... */ kprobe_post_handler_t post_handler; /* * ... called if executing addr causes a fault (eg. page fault). * Return 1 if it handled fault, otherwise kernel will see it. */ kprobe_fault_handler_t fault_handler; /* Saved opcode (which has been replaced with breakpoint) */ kprobe_opcode_t opcode; /* copy of the original instruction */ struct arch_specific_insn ainsn; /* * Indicates various status flags. * Protected by kprobe_mutex after this kprobe is registered. */ u32 flags;};For the kprobe execution process, the pre_handler and post_handler are quite important. Typically, when registering a kprobe, these two or one of them will also be filled in the structure.
Let’s randomly find a function to see the hook points reserved for kprobe during compilation. It can be seen that 5 bytes are reserved here for jump hijacking:
Next, let’s find a function that has registered a kprobe to examine:
It can be seen that the first five bytes have indeed been replaced with a call to a segment of shellcode. Let’s continue to examine the definition of this shellcode:
0xffffffffc0511000: pushf 0xffffffffc0511001: sub $0xa8,%rsp 0xffffffffc0511008: mov %rax,0x50(%rsp) 0xffffffffc051100d: mov %rcx,0x58(%rsp) 0xffffffffc0511012: mov %rdx,0x60(%rsp) 0xffffffffc0511017: mov %rsi,0x68(%rsp) 0xffffffffc051101c: mov %rdi,0x70(%rsp) 0xffffffffc0511021: mov %r8,0x48(%rsp) 0xffffffffc0511026: mov %r9,0x40(%rsp) 0xffffffffc051102b: mov %rbp,%rdx 0xffffffffc051102e: mov %rdx,0x20(%rsp) 0xffffffffc0511033: mov 0xb8(%rsp),%rsi 0xffffffffc051103b: mov 0xb0(%rsp),%rdi 0xffffffffc0511043: mov %rdi,0x80(%rsp) # Here is an interesting point, regs->rip equals the return address 0xffffffffc051104b: sub $0x5,%rdi # When passing to the subroutine, the address -5 0xffffffffc051104f: mov 0xd7(%rip),%rdx # 0xffffffffc051112d 0xffffffffc0511056: mov %r15,(%rsp) 0xffffffffc051105a: mov %r14,0x8(%rsp) 0xffffffffc051105f: mov %r13,0x10(%rsp) 0xffffffffc0511064: mov %r12,0x18(%rsp) 0xffffffffc0511069: mov %r11,0x30(%rsp) 0xffffffffc051106e: mov %r10,0x38(%rsp) 0xffffffffc0511073: mov %rbx,0x28(%rsp) 0xffffffffc0511078: mov 0xa8(%rsp),%rcx 0xffffffffc0511080: mov %rcx,0x90(%rsp) 0xffffffffc0511088: mov $0x18,%rcx 0xffffffffc051108f: mov %rcx,0xa0(%rsp) 0xffffffffc0511097: mov $0x10,%rcx 0xffffffffc051109e: mov %rcx,0x88(%rsp) 0xffffffffc05110a6: lea 0xb8(%rsp),%rcx 0xffffffffc05110ae: mov %rcx,0x98(%rsp) 0xffffffffc05110b6: lea (%rsp),%rcx 0xffffffffc05110ba: call 0xffffffff9b793140 <ftrace_ops_assist_func> 0xffffffffc05110bf: mov 0x90(%rsp),%rax 0xffffffffc05110c7: mov %rax,0xa8(%rsp) 0xffffffffc05110cf: mov 0x80(%rsp),%rax 0xffffffffc05110d7: mov %rax,0xb0(%rsp) 0xffffffffc05110df: mov (%rsp),%r15 0xffffffffc05110e3: mov 0x8(%rsp),%r14 0xffffffffc05110e8: mov 0x10(%rsp),%r13 0xffffffffc05110ed: mov 0x18(%rsp),%r12 0xffffffffc05110f2: mov 0x38(%rsp),%r10 0xffffffffc05110f7: mov 0x28(%rsp),%rbx 0xffffffffc05110fc: mov 0x40(%rsp),%r9 0xffffffffc0511101: mov 0x48(%rsp),%r8 0xffffffffc0511106: mov 0x70(%rsp),%rdi 0xffffffffc051110b: mov 0x68(%rsp),%rsi 0xffffffffc0511110: mov 0x60(%rsp),%rdx 0xffffffffc0511115: mov 0x58(%rsp),%rcx 0xffffffffc051111a: mov 0x50(%rsp),%rax 0xffffffffc051111f: mov 0x20(%rsp),%rbp 0xffffffffc0511124: add $0xa8,%rsp 0xffffffffc051112b: popf 0xffffffffc051112c: retIt can be seen that the first half and the second half of this shellcode are responsible for saving and restoring the register contents, while the middle part calls the function ftrace_ops_assist_func. Let’s continue to examine the definition of this function:
static void ftrace_ops_assist_func(unsigned long ip, unsigned long parent_ip, struct ftrace_ops *op, struct pt_regs *regs){ int bit; if ((op->flags & FTRACE_OPS_FL_RCU) && !rcu_is_watching()) return; bit = trace_test_and_set_recursion(TRACE_LIST_START, TRACE_LIST_MAX); if (bit < 0) return; preempt_disable_notrace(); op->func(ip, parent_ip, op, regs); preempt_enable_notrace(); trace_clear_recursion(bit);}Based on the content of the shellcode, let’s interpret these four parameters:
ip: rdi register, contains the starting address of the hijacked function, which here retrieves the return address of this call and subtracts 5, that is, the starting address of the function security_path_chmod. parent_ip: rsi register, contains the return address of the previous function, which is the return address of the call to security_path_chmod, also retrieved from the stack, as the shellcode source code clearly operates on stack instructions. op: rdx register, this is interesting, as it is calculated based on 0xd7(%rip), which means that all kprobe hijacking points use the same one. regs: rcx register, contains the content saved after the sub rsp in the shellcode.(Indeed, this is different from the conventional use of fastcall where rcx, rdx, r8, r9 are used)
Next, let’s take a look at the content pointed to by the ftrace_ops structure at the global address, as its content determines the next jump of the function:
crash> rd 0xffffffffc051112dffffffffc051112d: ffffffff9ce7f4e0 ........crash> struct ftrace_ops ffffffff9ce7f4e0 -xstruct ftrace_ops { func = 0xffffffff9b65a090 <kprobe_ftrace_handler>, next = 0xffffffff9ce7fe60 <ftrace_list_end>, flags = 0x1845, private = 0x0, ......}It can be seen that the next jump function is kprobe_ftrace_handler. Let’s continue to examine the definition of this function:
/* Ftrace callback handler for kprobes -- called under preempt disabled */void kprobe_ftrace_handler(unsigned long ip, unsigned long parent_ip, struct ftrace_ops *ops, struct pt_regs *regs){ struct kprobe *p; struct kprobe_ctlblk *kcb; /* Preempt is disabled by ftrace */ p = get_kprobe((kprobe_opcode_t *)ip); if (unlikely(!p) || kprobe_disabled(p)) return; kcb = get_kprobe_ctlblk(); if (kprobe_running()) { kprobes_inc_nmissed_count(p); } else { unsigned long orig_ip = regs->ip; /* Kprobe handler expects regs->ip = ip + 1 as breakpoint hit */ regs->ip = ip + sizeof(kprobe_opcode_t); __this_cpu_write(current_kprobe, p); kcb->kprobe_status = KPROBE_HIT_ACTIVE; if (!p->pre_handler || !p->pre_handler(p, regs)) { /* * Emulate singlestep (and also recover regs->ip) * as if there is a 5byte nop */ regs->ip = (unsigned long)p->addr + MCOUNT_INSN_SIZE; if (unlikely(p->post_handler)) { kcb->kprobe_status = KPROBE_HIT_SSDONE; p->post_handler(p, regs, 0); } regs->ip = orig_ip; } /* * If pre_handler returns !0, it changes regs->ip. We have to * skip emulating post_handler. */ __this_cpu_write(current_kprobe, NULL); }}This function is relatively simple, as it calls the pre and post functions filled in during the registration of the kprobe.
Interestingly, in the pre and post functions, the value of regs->ip is different. For the function security_path_chmod, for example:
In the pre-filter function, ip=security_path_chmod+1;
In the post-filter function, ip=security_path_chmod+5.
The kprobe process is like this. Next, let’s record something even more interesting: kretprobe:
struct kretprobe { struct kprobe kp; kretprobe_handler_t handler; kretprobe_handler_t entry_handler; int maxactive; int nmissed; size_t data_size; struct hlist_head free_instances; raw_spinlock_t lock;};It can be seen that it wraps a kprobe structure and also provides two handlers (entry == pre, handler == post). When registering a kretprobe, the main fields to fill in are these two;
It also provides a field called maxactive for concurrent processing when multiple tasks access this kprobe simultaneously. For each maxactive, a kretprobe_instance structure will be allocated and inserted into the free_instances linked list:
The code in the first red box will set kretprobe->kp.pre_handler to a fixed system function pre_handler_kretprobe; this actually shows that kretprobe relies on the logic of kprobe. During the distribution of a kretprobe, the first function executed is its kprobe’s pre_handler function.The second red box shows that the corresponding kretprobe_instance is allocated based on the value of maxactive and inserted into the free_instances linked list.
In the system function pre_handler_kretprobe, every time this function is entered, a kretprobe_instance structure is taken from free_instances to record the current information, and the current kretprobe_instance is inserted into the global linked list kretprobe_inst_table. Then, the entry_handler function is called, which is also triggered before calling the hijacked function:
Next, let’s see how it enters the “post process after calling the original function”. Here, unlike our conventional understanding of callbacks in Windows, which is call pre -> call original function -> call post:
It can be seen that in the arch_prepare_kreprobe function, the return address on the stack when calling the original function is replaced with a fixed system function kretprobe_trampoline.
In the following call stack example, the saved return address vfs_read+99 on the stack will be replaced with the function kretprobe_trampoline, thus hijacking the return position of the original function:
#8 [ffff8b969eedbda8] kprobe_ftrace_handler at ffffffff9b65a122 #9 [ffff8b969eedbdd8] ftrace_ops_assist_func at ffffffff9b7931da#10 [ffff8b969eedbe90] security_file_permission at ffffffff9bab2e81#11 [ffff8b969eedbec0] security_file_permission at ffffffff9bab2e85#12 [ffff8b969eedbec8] vfs_read at ffffffff9b8ccbc3Next, let’s take a look at the definition of the fixed hijacking functionkretprobe_trampoline:
0xffffffff9b6586b0 <kretprobe_trampoline>: push %rsp ; Note that here the push %rsp will have an important effect0xffffffff9b6586b1 <kretprobe_trampoline+1>: pushf 0xffffffff9b6586b2 <kretprobe_trampoline+2>: sub $0x18,%rsp0xffffffff9b6586b6 <kretprobe_trampoline+6>: push %rdi0xffffffff9b6586b7 <kretprobe_trampoline+7>: push %rsi0xffffffff9b6586b8 <kretprobe_trampoline+8>: push %rdx0xffffffff9b6586b9 <kretprobe_trampoline+9>: push %rcx0xffffffff9b6586ba <kretprobe_trampoline+10>: push %rax0xffffffff9b6586bb <kretprobe_trampoline+11>: push %r80xffffffff9b6586bd <kretprobe_trampoline+13>: push %r90xffffffff9b6586bf <kretprobe_trampoline+15>: push %r100xffffffff9b6586c1 <kretprobe_trampoline+17>: push %r110xffffffff9b6586c3 <kretprobe_trampoline+19>: push %rbx0xffffffff9b6586c4 <kretprobe_trampoline+20>: push %rbp0xffffffff9b6586c5 <kretprobe_trampoline+21>: push %r120xffffffff9b6586c7 <kretprobe_trampoline+23>: push %r130xffffffff9b6586c9 <kretprobe_trampoline+25>: push %r140xffffffff9b6586cb <kretprobe_trampoline+27>: push %r150xffffffff9b6586cd <kretprobe_trampoline+29>: mov %rsp,%rdi0xffffffff9b6586d0 <kretprobe_trampoline+32>: call 0xffffffff9b658910 <trampoline_handler>0xffffffff9b6586d5 <kretprobe_trampoline+37>: mov %rax,0x98(%rsp)0xffffffff9b6586dd <kretprobe_trampoline+45>: pop %r150xffffffff9b6586df <kretprobe_trampoline+47>: pop %r140xffffffff9b6586e1 <kretprobe_trampoline+49>: pop %r130xffffffff9b6586e3 <kretprobe_trampoline+51>: pop %r120xffffffff9b6586e5 <kretprobe_trampoline+53>: pop %rbp0xffffffff9b6586e6 <kretprobe_trampoline+54>: pop %rbx0xffffffff9b6586e7 <kretprobe_trampoline+55>: pop %r110xffffffff9b6586e9 <kretprobe_trampoline+57>: pop %r100xffffffff9b6586eb <kretprobe_trampoline+59>: pop %r90xffffffff9b6586ed <kretprobe_trampoline+61>: pop %r80xffffffff9b6586ef <kretprobe_trampoline+63>: pop %rax0xffffffff9b6586f0 <kretprobe_trampoline+64>: pop %rcx0xffffffff9b6586f1 <kretprobe_trampoline+65>: pop %rdx0xffffffff9b6586f2 <kretprobe_trampoline+66>: pop %rsi0xffffffff9b6586f3 <kretprobe_trampoline+67>: pop %rdi0xffffffff9b6586f4 <kretprobe_trampoline+68>: add $0x18,%rsp0xffffffff9b6586f8 <kretprobe_trampoline+72>: popf 0xffffffff9b6586f9 <kretprobe_trampoline+73>: ret From this function code, it can be seen that it will continue executing at the original function’s return value, such as vfs_read+99, and call the function trampoline_handler. After obtaining the return value of the function trampoline_handler, it will place it on the stack (push %rsp) and restore the registers until it returns to that return address.
The definition of the trampoline_handler function is also relatively simple:
1. Find the current task’s inserted item from the global linked list kretprobe_inst_table and retrieve the structure kretprobe_instance.
2. Find the return address before it was replaced by the function arch_prepare_kretprobe.
3. Execute the post function.
4. Insert the current kretprobe_instance back into free_instances:
Conclusion:
This note does not provide in-depth analysis or introduce little-known details; it merely serves as an understanding of one of the kernel hook interfaces in Linux, recorded and shared from the perspective of a Windows kernel developer, providing a different approach compared to Windows kernel callbacks. I hope everyone finds it interesting~