Assembly language is the foundation for learning reverse engineering. This article analyzes function calls from the assembly level to understand the specific implementation processes of stack pushing, jumping, execution, and returning, as well as the application of the stack.

My knowledge is limited, so please point out any errors or unclear points.

Your encouragement is the motivation for me to write articles.

Before introducing the specific process of function calls, let’s first understand a few key points.

• 1.1 Execution Flow of a Program

Programs are executed sequentially. How does the CPU implement this?

The execution of a program relies on a key register.

First, let’s understand the concept of the Program Counter.

According to Baidu Baike:

As can be seen, the function of the Program Counter is to store the address of the next instruction to be executed.

In X86 assembly, the register that performs the program counter function is called EIP, the Instruction Pointer register.

We can write a simple program to examine the disassembly.

The address stored in the EIP register is the address of the instruction that is about to be executed.

• 1.2 Stack Structure of a Program

In classic operating systems, the stack always grows downwards. The top of the stack is located by the ESP register. The push operation decreases the address of the top of the stack, while the pop operation increases it.

Function calls rely on the stack, which stores all the information during the function call.

• Return address and parameters of the function;

• Non-static local variables of the function;

• Context preservation: saving some registers that do not need to be changed.

To correctly handle the stack balance during function calls, we need to adjust the positions of two registers, EBP and ESP:

1. Raise EBP to the position of ESP.

2. The value of ESP changes dynamically; as more temporary variables are allocated, ESP will continue to decrease.

3. The structure diagram of the stack after calling the function.

• 1.3 Calling Conventions and Stack Balance

During the function call process, parameters need to be pushed onto the stack in advance, and after the function call ends, the previously pushed parameters also need to be popped from the stack.

This raises a question: Should the work of popping from the stack be done by the caller or the called function?

This requires an agreement.

Common stack balance methods for calling conventions:

Having understood the basic concepts, let’s return to the topic: function calls.

• Push onto stack: function parameters pushed onto stack, return address pushed onto stack

• Jump: jump to the code location of the function to execute

• Execute: execute the function code

• Return: balance the stack, find the return address and jump back

• 2.1 Call Instruction

0x210000 call swap;
0x210005 mov ecx,eax;

The call instruction can be broken down into two instructions:

push 0x210005;
jmp swap;

First, push the address of the next instruction onto the stack; the purpose is to jump back after the function call is complete;

Then modify EIP to jump to the function’s address.

• 2.2 Ret Instruction

The ret instruction indicates that the current top value of the stack is taken as the return address, modifying EIP to that address.

After executing this step, a basic function call is completed.

Test Program

In the main function, call a function that swaps two numbers.

#include <stdio.h>


void _stdcall swap(int&a,int&b);


// Function to swap two numbers
void _stdcall swap(int&a, int&b)
{
    int c = a;
    a = b;
    b = c;
}


void main()
{

    int a = 1;
    int b = 2;
    printf("Before swap: a =%d,b=%d\n",a,b);
    swap(a,b);
    printf("After swap: a= %d,b=%d\n",a,b);

    getchar();

}

The program’s output:

• 3.1 Stack Pushing

• 3.1.1 Function Parameters Pushed onto Stack

Before actually calling the function, the parameters are pushed onto the stack.

We observe the changes through registers and memory:

It is found that the parameters have been pushed onto the stack.

• 3.1.2 Return Address Pushed onto Stack

After entering the swap function with F11, a segment of data appears in memory; what is it?

It is actually the address of the next instruction of the call instruction:

• 3.2 Jump

Jump to the internal swap function with detailed annotations below.

• 3.3 Return

Before executing ret.

After executing ret, please slide the left side to fill the puzzle first:

Since the __stdcall calling convention is used, stack balancing is performed inside the called function.

Thus, the implementation of a function call has been analyzed. Many details have not been elaborated upon. This article serves to summarize key points of assembly, hoping to meet like-minded individuals.

The process of becoming a reverse engineering expert is arduous; you have already taken the first step. Keep it up.

Kanxue ID: Jabez

https://bbs.pediy.com/user-825190.htm

This article is original by Kanxue Forum Jabez

Reprint must indicate the source from the Kanxue Community

Book Recommendations:

• “Encryption and Decryption”

This is a must-read book for reverse engineering. It discusses the basic knowledge and skills related to software security from the perspective of software reverse engineering, making it one of the essential reads for security professionals.

Kanxue recommends the following transitional books before “Decryption”:

• “Assembly Language: Based on x86 Processors” by Kip Irvine

• “Assembly Language” by Wang Shuang

• “Programming Windows” by Charles Petzold (explained using VC)

Long press to recognize the QR code to purchase the above books

Recommended Popular Technical Articles:

Click the original text to see what everyone is saying?