Introduction to White-Box Encryption for Android Applications

0x00 Background

The expert ys often mentions white-box encryption, but I never quite understood what it is. I took some time to summarize the principles of white-box cryptography on Android and related case analyses of cracking.

White-Box Attacks

A white-box attacker has complete control over the terminal, allowing them to observe and modify the program’s internal data during runtime, including memory information, disk read/write permissions, etc. They can dump data from memory during execution, making the keys stored in memory insecure. This attack environment is called a white-box attack.

Devices with the highest permissions like Android and iOS (after rooting or jailbreaking) are considered white-box attack environments.

Common tools for white-box attack analysis include JEB, IDA Pro, OllyDbg, VMware, Hopper, etc.

White-Box Cryptography

White-box cryptography refers to a special type of encryption method that can resist attacks in a white-box environment. Its goal is to ensure that the complete key does not appear in the runtime environment. To achieve this, the key itself needs to be obfuscated as much as possible while providing functions to handle it.

For example, if the plaintext key is abcd, the obfuscated key can turn into a very complex matrix. Based on an algorithm, four intermediate keys are generated each time: aefg, hbigk, lmcn, opqrstd. Using these keys, four ciphertexts are generated, and then these four segments of ciphertext are combined into a single ciphertext through an algorithm. This process can be repeated until the final ciphertext is generated. Throughout this process, the original key never appears in memory, significantly increasing the difficulty for white-box attackers to obtain the key.

0x01 Pain Points of Mobile Storage

In business, it may be necessary to store sensitive data such as keys, user tokens, and emails locally on Android. Mobile apps operate in a white-box attack environment, which allows for debugging, memory dumping, secondary modifications, packet analysis, etc. Sensitive information in the app, both local and transmitted data, can be easily reverse-engineered.

1.1 Common Insecure Storage Methods for Keys on Mobile:

1. Keys are directly hardcoded in Java code, making them easy to reverse-engineer into Java code.
2. Keys are stored in plaintext in the private directory’s sharedprefs file, which can be easily exported and viewed on rooted devices.
3. Keys are split into different segments, some stored in files and others in code, then concatenated. Although this makes the operation complex, it is still on the Java layer, and reverse engineers can easily reverse-engineer it with some time.
4. Using NDK development, placing keys in so files, with encryption and decryption operations in so files. This increases security to some extent and blocks some reverse engineers, but experienced reverse engineers can still use IDA to crack it.
5. Not storing keys in so files, but performing encryption and decryption operations in the so file, naming the encrypted key something ordinary, and storing it in the assets directory or elsewhere. Adding irrelevant code (dummy instructions) in the so file can increase static analysis difficulty, but dynamic debugging methods can still trace encryption and decryption functions to find key contents.

1.2 How to Solve the Key Security Storage Problem?

Android Provides a Keystore System:

The Android system has a security mechanism called Keystore that protects key materials from unauthorized use. https://developer.android.com/training/articles/keystore.html?hl=en

High-Version Hardware Security Module TEE:

Supported devices running Android 9 (API level 28) or higher can use StrongBox Keymaster, which is an implementation of the Keymaster HAL located in the hardware security module. It includes the following components:

Own CPU.
Secure storage space.
True random number generator.
Additional mechanisms to resist software tampering and unauthorized application loading.

Hardware encryption is highly resistant to cracking and supports a variety of algorithms.

However, low-version Android Keystore has obvious shortcomings:

The AndroidKeyStore provider introduced in Android 4.3 (API level 18). The APIs called include KeyStore, KeyPairGenerator, and KeyGenerator classes, etc. However, the variety of supported algorithms is limited. KeyStoreEncryDemo: demo https://github.com/zhibuyu/KeyStoreEncryDemo

This security mechanism still stores the key files in the /data/misc/keystore/user_0/ directory, making them vulnerable to cracking and exploitation. For example, when an attacker has root access, they can copy other apps’ keys using the cp, chown commands. Just changing the UID to their own app’s UID allows them to steal other apps’ keys.

Cracking case: Using root privileges to steal keys from Android Keystore of other apps: https://bbs.pediy.com/thread-222890.htm

Best Solutions in the Industry

Due to the limited algorithms supported by Android Keystore and its inability to downgrade support for low-version Android devices, it cannot be used directly in business applications. Additionally, the issue of exposing keys in external business applications is virtually unsolvable. Therefore, many companies typically choose white-box encryption in conjunction with hardening and OLLVM obfuscation to increase the reverse engineering costs for attackers.

0x03 Introduction to White-Box Encryption Technology

White-box encryption technology needs to resist white-box attacks, and its core idea is to hide the key so that the key value does not appear in memory during the encryption execution process.

There are two strategies for constructing white-box cryptography: standard cryptographic algorithm white-boxing and constructing entirely new cryptographic algorithms. The commonly used technique now is lookup table technology, which hides the key in a lookup table.

Standard cryptographic algorithm white-boxing, based on the security theory of standard cryptographic algorithms and without changing the original algorithm’s functions, designs the original cryptographic algorithm using white-box cryptography technology to effectively ensure its key security in a white-box attack environment.
Constructing entirely new cryptographic algorithms, the cryptanalysis results of the new algorithm must not be weaker than those of standard cryptographic algorithms while also possessing the ability to resist white-box attacks.

0. Introduction to Cryptography

Cryptographic algorithms are steps used to solve complex problems, commonly referred to as algorithms. The steps to generate ciphertext from plaintext, known as the encryption process, are called encryption algorithms, while the steps for decryption are called decryption algorithms. Together, encryption and decryption algorithms are referred to as cryptographic algorithms.

Keys are required in cryptographic algorithms (key). According to Kerckhoffs’s principle, the cryptographic algorithm is standard, while the key must be kept secret; thus, the security of the key is crucial for the cryptographic system.

Symmetric Cryptographic Algorithms share a key between both communicating parties for encrypting data blocks or streams of any size, including messages, files, encryption keys, and passwords. Common symmetric cryptographic algorithms include DES and AES.

Asymmetric Cryptographic Algorithms are also known as public-key cryptography, where different keys are used for encryption and decryption, namely the private key and the public key. Public-key cryptographic algorithms are often used to encrypt small data blocks, such as encryption keys or hash values used in digital signatures. Common public-key cryptographic algorithms include RSA and ECC.

Cryptographic Protocols refer to applications of cryptographic algorithms. Common protocols include SSL/TLS, HTTPS.

Other Cryptographic Technologies include hash functions, MAC, digital signatures, digital certificates, and random number generators.

Practical Cryptography for Developers: https://cryptobook.nakov.com/

1. Implementation of White-Box Cryptography

White-box cryptography can be divided into two categories based on implementation: static white-box and dynamic white-box.

Static white-box refers to a cryptographic algorithm combined with a specific key processed by white-box cryptography technology to form a specific cryptographic algorithm library, known as a white-box library. The white-box library possesses specific cryptographic functionalities (encryption, decryption, and both) and can effectively protect the original key’s security in a white-box attack environment. Updating the static white-box requires regenerating the white-box library.

Dynamic white-box means that once the white-box library is generated, it does not need to be updated again. The original key is transformed into a white-box key using the same white-box cryptography technology. The white-box key can perform normal encryption or decryption functions when passed into the corresponding white-box library. The white-box key is secure, and attackers cannot obtain any information about the original key through analysis of the white-box key.

Introduction to White-Box Encryption for Android Applications — Insert image description here

2. Case Analysis of White-Box Encryption

Common white-box algorithms include: White-Box AES, White-Box SMS4.

A simple implementation of White-Box SMS4 can be found here: https://github.com/ohhoo/White-box-Cryptographic/blob/master/wbSM4_Shang/wbSM4.cpp
A simple implementation of White-Box AES can be found here: https://github.com/Gr1zz/WhiteBoxAES

AES is one of the most commonly used symmetric encryption algorithms in our daily development, mainly implemented through the following four processes:

1. S-box byte substitution (SubBytes)
2. Row shifting (ShiftRows)
3. Column mixing (MixColumns)
4. Round key addition (AddRoundKey)

AES-128 typically undergoes 10 rounds of transformation, and the encryption and decryption processes are as follows:

3. Countermeasures Against White-Box Encryption

3.1 Direct Black-Box Invocation of Binaries

Counteracting encryption does not necessarily require reverse-engineering the algorithm; it can be done by directly invoking the black-box. Obtaining the results after encryption signing is sufficient. Especially after applying OLLVM obfuscation to the encryption algorithm, the costs of debugging and reverse-engineering become prohibitively high. Typically, Xposed or Frida can be used to reflect the signing function and set up RPC services or execute through the Unicorn emulator.

1. Frida hook for direct black-box invocation

Invocation case: https://github.com/zhaoboy9692/flask_frida_rpc

Frida code hook for analyzing XX’s encryption class function:

var result;

function callEleMeFun(url_path) { // Define export function    Java.perform(function () {        var a = Java.use('class'); // This part needs to be modified, you need to find XX's encryption class        var context = Java.use('android.app.ActivityThread').currentApplication().getApplicationContext();        var res = a.sneer(context, "4f93e640-5c6f-4980-bd3d-c1256672a64d", url_path, "a");        result = {"ex_r": res[0], "ex_dc": res[1], "ex_d": res[2]};    });    return result;}

// Frida provides RPC service, exposing signing function
rpc.exports = {    callsecretfunctioneleme: callEleMeFun,};

2. Simulated Execution with Xposed + Server

Previously, black-box invocation was generally implemented by building an Android server using AndServer + Service to call the so file: https://links.jianshu.com/go?to=https%3A%2F%2Fmp.weixin.qq.com%2Fs%2FDo4rGrMNFGx5HqFKM9vA1g. Generally, it requires hooking the calling function with Xposed or opening the key so and calling specific function calls, and using AndServer to set up a server for signing services.

Based on long connections and code injection, the Android private API exposure framework Sekiro has set up basic exposure services, and we can easily implement an encryption service with just a few lines of code: https://github.com/virjar/sekiro

Example code:

import de.robv.android.xposed.IXposedHookLoadPackage;
import de.robv.android.xposed.XC_MethodHook;
import de.robv.android.xposed.XposedBridge;
import de.robv.android.xposed.XposedHelpers;
import de.robv.android.xposed.callbacks.XC_LoadPackage;

public class HookMain implements IXposedHookLoadPackage {    public static XC_LoadPackage.LoadPackageParam loadPackageParam = null;

    @Override    public void handleLoadPackage(XC_LoadPackage.LoadPackageParam lpparam) throws Throwable {        if (lpparam.packageName.equals("com.ss.XXXXX")) {            HookMain.loadPackageParam = lpparam;            try {

                // Inject service into com.ss.android.ugc.aweme.splash.SplashActivity -> onCreate
                XposedHelpers.findAndHookMethod("com.ss.XXXXX", lpparam.classLoader, "onCreate", Bundle.class, new XC_MethodHook() {                    @Override                    protected void afterHookedMethod(MethodHookParam param) throws Throwable {                        super.afterHookedMethod(param);                        final SekiroClient sekiroClient = SekiroClient.start("sekiro.virjar.com", "client-6", "group-6"); // Create Sekiro client                        sekiroClient.registerHandler("user_search", new UserSearchHandler()); // Register and start service                        XposedBridge.log("=========== Sekiro service started successfully ===========");                    }                });            } catch (Exception e) {                XposedBridge.log("=========== Sekiro service failed to start ===========");            }

        }    }}

The key lies in these two lines:

                        final SekiroClient sekiroClient = SekiroClient.start("sekiro.virjar.com", "client-6", "group-6"); // Create Sekiro client                        sekiroClient.registerHandler("user_search", new UserSearchHandler()); // Register and start service                        XposedBridge.log("=========== Sekiro service started successfully ===========");

By creating a Sekiro client and registering the handler processing function, the service will automatically start.

import com.example.demo.hook.HookMain;
import com.google.gson.Gson;
import com.virjar.sekiro.api.SekiroRequest;
import com.virjar.sekiro.api.SekiroRequestHandler;
import com.virjar.sekiro.api.SekiroResponse;

import de.robv.android.xposed.XposedHelpers;

public class DouYinUserSearchHandler implements SekiroRequestHandler {


    @Override    public void handleRequest(SekiroRequest sekiroRequest, SekiroResponse sekiroResponse) {        Gson gson = new Gson();

        String query = sekiroRequest.getString("query");        String start = sekiroRequest.getString("start", "0");        String count = sekiroRequest.getString("count", "10");

        long param2 = Long.parseLong(start);        int param3 = Integer.parseInt(count);

        if (query == null || query.equals("")) {            sekiroResponse.send("Please provide the required parameter: query");        }

        Class SearchApi = XposedHelpers.findClass("com.ss.XXXXXX.discover.api.SearchApi", HookMain.loadPackageParam.classLoader);        Object object = XposedHelpers.callStaticMethod(SearchApi, "a", query, param2, param3);        sekiroResponse.send(gson.toJson(object));    }}

The handler code logic uses an API to pass input function parameters, reflectively calling the function to process and return results.

Results can be accessed directly via http://sekiro.virjar.com/invoke?group=sekiro-6&action=XXX&query=XXX.

3. Simulated Execution

By cross-platform simulating Android native library functions: https://github.com/zhkl0228/unidbg. The details of simulating execution can be analyzed in follow-up articles.

3.2 Error Injection to Crack the Original Key

Core Idea:

This article from quarkslab: https://blog.quarkslab.com/differential-fault-analysis-on-white-box-aes-implementations.html

Principle: Although the white-box AES program does not reveal the key during execution, it has been shown that the original key can be derived from intermediate keys: https://github.com/SideChannelMarvels/Stark

For white-box AES-128, providing a round key and its index value is sufficient to derive the original encryption key.

Now the question is, how to obtain the intermediate keys?

This can be done by using DFA to statically generate Fault data, then calling [phoenixAES] to restore the key. (Forgive me for not understanding the specific implementation details of DFA) See: https://github.com/SideChannelMarvels/Deadpool/blob/master/README_dfa.md

The phoenixAES algorithm can be found here: https://github.com/SideChannelMarvels/JeanGrey/tree/master/phoenixAES. After executing phoenixAES, the last round key is generally obtained. Then use the aes_keyschedule above to restore the key.

Cracking Case: You can refer to this article shared by Master Xue: https://bbs.pediy.com/thread-254042.htm (not through static DFA, but using IDA dynamic patching to obtain Fault data)

1. To restore the key of white-box AES through DFA, you only need to find a way to construct a reasonable Fault data set, which can then be input into the implementation of phoenixAES to restore the key.

2. When constructing Fault data, there are generally two methods: * The method given by the official for static changes, which is fully automated and does not require any binary code analysis. * The IDA dynamic patching method, which requires code analysis to find the right timing for the patch.

3. During the dynamic patching process, pay attention to the output data. If the output only changes 1 byte, it means the patch is too late; if the output has 16 bytes different, it means the patch is too early; if the output changes exactly 4 bytes, and those 4 bytes' positions also meet the criteria, then the timing is right.

Another solution recommended by experts:

Capture the binary of the key function, then run it with Unicorn, inputting random plaintext (ciphertext), and capture memory and register changes, finally using side-channel methods to derive the key.

4. Advantages and Disadvantages of White-Box Cryptography

Advantages:

Due to the merging of the algorithm and the key, it effectively hides the key while obfuscating the encryption logic. Specifically, one implementation approach of white-box encryption is to completely replace the algorithm with a lookup table, as both the algorithm and the encryption key are known. Thus, the algorithm and key are solidified into a lookup table representation, which is the implementation process of white-box keys.

Disadvantages:

There are pros and cons; white-box encryption requires solidifying the encryption and decryption algorithms into tables, which inevitably increases space overhead, as a series of tables must be stored. In most cases, this means trading time for space, requiring optimization of the specific implementation algorithms, essentially breaking down a large table into several smaller tables, which increases the lookup times (time overhead) but reduces the space occupied by the tables.

5. Conclusion

This article provides an understanding of the reasons for using white-box encryption in Android applications, a basic introduction to white-box cryptography, and related encryption cracking methods and key restoration operations. I hope friends reading this article gain something!

References:

https://www.zhihu.com/question/35136485/answer/84491440

Android pure local security storage solutions: https://richardcao.me/2019/03/14/Android-Local-Security-Storage/

Android’s own key store system: https://www.jianshu.com/p/dc5a9f906eb8 Android keystore vulnerabilities: https://androidvulnerabilities.org/all

White-box cryptography technology: https://zhuanlan.zhihu.com/p/23295431

JD.com reinforcement: https://aks.jd.com/doc/mobiledoc/chan-pin-jian-jie.html

A Tutorial on White-box AES: https://blog.wxk.me/2019/07/05/wbac_tutorial/