Introduction

Is it becoming increasingly difficult to capture HTTPS packets on Android systems? In high versions, adding CA certificates is not possible, and packet capture software relies on too many dependencies. When using VPN mode or HOOK programs, they can be detected by apps. The cost of countermeasures is rising. Is there a universal tool available?

Yes, eCapture for Android^[1] is here. From now on, capturing HTTPS communication packets on Android no longer requires installing CA certificates, downloading a bunch of Python dependencies, repackaging SSL libraries, or modifying numerous phone parameters. It can be enabled with a single click, simple and clear.

Overview of eCapture

eCapture is a software that can capture HTTPS plaintext without requiring CA certificates. It supports pcapng format and can be viewed directly with Wireshark. Based on eBPF technology, it only requires root access to capture packets with one click. The Chinese name of eCapture is “旁观者” (Observer), meaning that those in power may be confused, but the observer sees clearly.

In early 2022, during the Shanghai pandemic, the author began writing and open-sourcing^[2] eCapture. It has been half a year, and it has already received 4200 stars on GitHub.

eCapture is a packet capture software based on eBPF technology, depending on whether the system kernel supports eBPF. It currently supports operating systems with X86_64 and ARM64 Linux kernel 4.18 and above, and ARM64 Android (Linux) kernel 5.4 and above. The latest version, v0.4.3, was released on September 9, 2022.

Demo Video

After downloading, start with a single command, clean and efficient. ./ecapture tls -w ecapture.pcapng

First, watch the demo video, which runs in an environment of Ubuntu 21.04 and Android 5.4 (Pixel 6).

Module Functions

eCapture supports information extraction and capture for modules such as tls, bash, mysqld, and postgres. This article will only discuss the tls module for capturing HTTPS/TLS plaintext.

Plaintext Capture of Encrypted Communication — TLS Module

The tls module supports libraries such as openssl, gnutls, nspr/nss, and boringssl for encrypted communication. However, on Android, the pcapng mode only supports boringssl, while text mode supports all.

How to Use eCapture

Environment Dependencies

1. Operating system: Linux kernel 4.18 and above, Android kernel 5.4 and above.
2. Supports BPF, optional support for BTF (varies by eCapture version)
3. Root access required

Version Selection

BPF CO-RE^[3] features a universal format for cross-kernel version compatibility. Some Android phones do not have BTF enabled. You can check the system configuration to confirm; CONFIG_DEBUG_INFO_BTF=y means enabled; CONFIG_DEBUG_INFO_BTF=n means disabled; others do not support BPF and cannot use eCapture.

cfc4n@vm-server:~$# cat /boot/config-`uname -r` | grep CONFIG_DEBUG_INFO_BTF
CONFIG_DEBUG_INFO_BTF=y

On Android systems, the config is gzip compressed, and the configuration file directory has also changed. You can use zcat /proc/config.gz instead.

eCapture is released with a version that supports CO-RE ELF binaries. The Android version will release a version that does not support BTF (i.e., no CO-RE). After downloading, you can confirm with ./ecapture -v. The version information of the non-CO-RE version includes the kernel version at compile time.

# no CO-RE
eCapture version:	linux_aarch64:0.4.2-20220906-fb34467:5.4.0-104-generic
# CO-RE
eCapture version:	linux_aarch64:0.4.2-20220906-fb34467:[CORE]

If the version does not meet your needs, you can compile it yourself, with steps found at the end of the article.

Global Parameter Introduction

Focus on the following global parameters:

root@vm-server:/home/cfc4n/# ecapture -h
      --hex[=false]  print byte strings as hex encoded strings
  -l, --log-file=""  -l save the packets to file
  -p, --pid=0   if pid is 0 then we target all pids
  -u, --uid=0   if uid is 0 then we target all users

1. --hex is used for stdout output scenarios, displaying results in hexadecimal, particularly necessary for viewing non-ASCII characters, especially in encrypted and encoded content.
2. -l, --log-file= saves the results to a file path.
3. -p, --pid=0 specifies the target process for capture, defaulting to 0 to capture all processes.
4. -u, --uid=0 specifies the target user for capture, defaulting to 0 to capture all users, which is very necessary for Android.

Module Parameters

root@vm-server:/home/cfc4n/project/ssldump# bin/ecapture tls -h
OPTIONS:
      --curl=""  curl or wget file path, use to detect openssl.so path, default:/usr/bin/curl
      --firefox="" firefox file path, default: /usr/lib/firefox/firefox.
      --gnutls="" libgnutls.so file path, will automatically find it from curl default.
      --gobin="" path to binary built with Go toolchain.
  -h, --help[=false] help for tls
  -i, --ifname="" (TC Classifier) Interface name on which the probe will be attached.
      --libssl="" libssl.so file path, will automatically find it from curl default.
      --nspr=""  libnspr44.so file path, will automatically find it from curl default.
      --port=443 port number to capture, default:443.
      --pthread="" libpthread.so file path, use to hook connect to capture socket FD.will automatically find it from curl.
      --wget=""  wget file path, default: /usr/bin/wget.
  -w, --write="" write the raw packets to file as pcapng format.

-i Parameter

The -i parameter specifies the name of the network interface, defaulting to eth0 on Linux and wlan0 on Android, which you can specify yourself.

Output Modes

The output format supports two formats: text and pcapng files. There are three parameters:

1. Default, global parameter, outputs text results to stdout.
2. -l global parameter, saves text results to a file path.
3. -w tls module parameter, saves pcapng results to a file path.

Library Paths

Linux supports various libraries, and the paths for different libraries vary.

Text Mode

Using -l or omitting the -w parameter will enable this mode.

It supports various TLS encryption libraries such as openssl, boringssl, gnutls, and nspr/nss. It supports all versions of encryption protocols including DTLS, TLS 1.0 to TLS 1.3. It supports all global filtering parameters like -p and -u.

Pcapng Mode

Enabling this mode requires the -w parameter, and you must choose the network interface name with -i. The default for Linux is eth0, and for Android, it is wlan0.

This mode only supports capturing data from openssl and boringssl libraries. TLS 1.3 protocol is currently not supported.

Library and Parameter Support

On Linux systems, most libraries and parameters are supported. However, on Android systems, due to kernel and ARM architecture reasons, there are some differences in supported parameters.

Parameter Support for Different Modes

The global parameters -p and -u support text mode but do not support pcapng mode. This is because pcapng mode is implemented using eBPF TC technology.

Library and Protocol Support for Different Modes

Pcapng mode does not currently support TLS 1.3, but the TLS 1.3 key capture feature^[4] has been developed, though it encounters some bugs that are still being resolved. The author is not an expert in openssl and is not very familiar with the TLS protocol. Additional knowledge in these areas is needed to solve the issues, and friends who are skilled in this area are welcome to join in.

Using with tcpdump

eCapture is based on eBPF TC for traffic capture and saves to pcapng files. It captures the TLS Master Secret using eBPF Uprobe. Based on Wireshark’s Decryption Secrets Block (DSB)^[5] standard, it implements the gopacket’s DSB function^[6] to merge network packets with keys and save them to pcapng.

eCapture is not as powerful as tcpdump for network packet capture and does not support a rich set of parameters. You can use eCapture to capture master secrets and use tcpdump to capture network packets, then use Wireshark to set the key file for custom settings.

Network Packet Capture

The common usage of tcpdump will not be elaborated here.

Key Capture

When enabling eCapture, whether in text or pcapng mode, it will save the TLS master secrets key data to ecapture_masterkey.log.

Viewing Network Packets

Open the network packet file with Wireshark, set this master key file, and you will be able to see the plaintext after TLS decryption.

Configuration path: Wireshark –> Preferences –> Protocols –> TLS –> (Pre)-Master-Secret log filename

Parameters

Specify Path

Default Path

On Android, Google uses the boringssl library, which is a wrapper around the C++ language based on libssl. By default, it will use the path /apex/com.android.conscrypt/lib64/libssl.so.

Confirming APP’s Library

You can use lsof -p {APP PID}|grep libssl to confirm. If it is not the default path, you can specify it using the --libssl parameter.

Advanced Usage

If you need to view an APP with a custom SSL library, you can modify eCapture yourself to achieve this.

Custom Function Names and Offsets

First, you need to determine the function name of the HOOK function or the symbol table address.

No Source Code

If you do not have the source code for the target library, you can use tools like IDA for static analysis or dynamic debugging to determine the address offset of SSL Write. Fill in the configuration in the user/module/probe_openssl.go file in the corresponding probe configuration section.

{
    Section:          "uprobe/SSL_write",
    EbpfFuncName:     "probe_entry_SSL_write",
    AttachToFuncName: "SSL_write",
    UprobeOffset:       0xFFFF00, // TODO
    BinaryPath:       binaryPath,
},

Automatic Offset Calculation

If you have the source code, you can read the source to determine the function name or the address in the symbol table. For reading member attributes of structures, you can automatically calculate using the offsetof macro. Read the content via offset.

//  g++ -I include/ -I src/ ./src/offset.c -o off
#include <stdio.h>
#include <stddef.h>
#include <ssl/internal.h>
#include <openssl/base.h>
#include <openssl/crypto.h>

#define SSL_STRUCT_OFFSETS               \
    X(ssl_st, session)              \
    X(ssl_st, s3)              \
    X(ssl_session_st, secret)        \
    X(ssl_session_st, secret_length)  \
    X(bssl::SSL3_STATE, client_random) \
    X(bssl::SSL_HANDSHAKE, new_session) \
    X(bssl::SSL_HANDSHAKE, early_session) \
    X(bssl::SSL3_STATE, hs) \
    X(bssl::SSL3_STATE, established_session) \
    X(bssl::SSL_HANDSHAKE, expected_client_finished_)    

struct offset_test
{
    /* data */
    int t1;
    bssl::UniquePtr<SSL_SESSION> session;
};

int main() {
    printf("typedef struct ssl_offsets { // DEF \n");
#define X(struct_name, field_name) \
    printf("   int " #struct_name "_" #field_name "; // DEF\n");
    SSL_STRUCT_OFFSETS
#undef X
    printf("} ssl_offsets; // DEF\n\n");

    printf("/* %s */\nssl_offsets openssl_offset_%d = { \n",
           OPENSSL_VERSION_TEXT, OPENSSL_VERSION_NUMBER);

#define X(struct_name, field_name)                         \
    printf("  ." #struct_name "_" #field_name " = %ld,\n", \
           offsetof(struct struct_name, field_name)); \
    SSL_STRUCT_OFFSETS
#undef X
    printf("};\n");
    return 0;
}

Parameter Extraction

For parameters, you need to confirm the types of the parameters of the HOOK function to determine the reading method. You can refer to the implementation of SSL_write function in kern/openssl_kern.c.

Compilation

ARM Linux Compilation

Most public cloud vendors provide ARM64 CPU servers. The author chose Tencent Cloud. In Guangzhou Six District, it is named Standard SR1 (SR1 is ARM 64 CPU), and the lowest configuration SR1.MEDIUM2 with 2 cores and 2G is sufficient for the compilation environment. It can be purchased on a pay-as-you-go basis, which is quite cost-effective.

Select the operating system ubuntu 20.04 arm64.

ubuntu@VM-0-5-ubuntu:~$sudo apt-get update
ubuntu@VM-0-5-ubuntu:~$sudo apt-get install --yes wget git golang build-essential pkgconf libelf-dev llvm-12 clang-12  linux-tools-generic linux-tools-common
ubuntu@VM-0-5-ubuntu:~$wget https://golang.google.cn/dl/go1.18.linux-arm64.tar.gz
ubuntu@VM-0-5-ubuntu:~$sudo rm -rf /usr/local/go && sudo tar -C /usr/local -xzf go1.18.linux-arm64.tar.gz
ubuntu@VM-0-5-ubuntu:~$for tool in "clang" "llc" "llvm-strip"
do
sudo rm -f /usr/bin/$tool
sudo ln -s /usr/bin/$tool-12 /usr/bin/$tool
done
ubuntu@VM-0-5-ubuntu:~$export GOPROXY=https://goproxy.cn
ubuntu@VM-0-5-ubuntu:~$export PATH=$PATH:/usr/local/go/bin

Compilation Methods

1. ANDROID=1 make command compiles the binary program that supports core version.
2. ANDROID=1 make nocore command compiles the binary program that only supports the current kernel version.

When Wireshark Meets eBPF

+ =

Contributors

Thanks to contributors such as chriskaliX, chenhengqi, vincentmli, huzai9527, yihong0618, sfx(4ft35t), and others. Special thanks to tiann Weishu for promoting this need in Android.

Recruitment

Meituan Information Security Department is directly recruiting a project team leader, with no middleman markup. Positions are available in Beijing or Shanghai, focusing on RASP, HIDS products, JVM, Linux kernel intrusion detection, and other security R&D positions.

1. Senior JAVA Development Engineer
2. Senior Golang Engineer
3. More positions can be found at https://www.cnxct.com/jobs/^[7]

References