This article is an excellent post from the Kxue Forum. Author ID: xwtwho
The original intention was to brush Douyin too much and find that I couldn’t directly search in the list of liked videos, so I wanted to implement it myself and record the process as a learning note, purely for technical exchange.
1. Software and Hardware Environment:
Douyin Android (V15.7, Application Treasure Channel)
Douyin Web (V16.1)
IDA 7.5
Frida 14.2.2
Gda3.86
JEB
jadx-gui
unidbg
LineageOS 17.1 (Android 10)
Xiaomi 8
2. Chronological Record (Directly in chronological order, writing down the pitfalls)
1. App X-Gorgon Algorithm Positioning
At first, I searched online and found the following:
https://blog.csdn.net/weixin_48271161/article/details/108544446
In this V15.7 version, I couldn’t find the related so (later found that the version I looked at corresponded to libmetasec_ml.so).
There were some other searches, like searching for X-Gorgon, hooking HashMap, but they didn’t work well for this version, so I started from scratch to find this encryption key point.
First, start Frida and output the JNI functions:
Looking back now, I found that I had output the libmetasec_ml.so JNI records (although not the direct encryption function), but at that time there were probably too many records, and I didn’t notice it. I probably didn’t pay attention even if I saw it because I didn’t know what this so was for at first.
Seeing this Cronet, I searched for it:
Cronet Network Library Series (1): Use Cases and Principle Implementation Details
https://segmentfault.com/a/1190000021095757
https://blog.csdn.net/u010983881/article/details/97770544/
[Android] Mobile Access to Cronet Practice
Based on the login URL, I found the following code:
Then I browsed the code based on the call relationship, looking for Cronet related calls, browsing the code, and flipping to com.ttnet.org.chromium.net.impl.CronetUrlRequest, but I didn’t find a place to set the X-Gorgon header (other headers were set).
In the middle, I also tried to reverse the call through send, but the call line was too long, so I decided to try another method.
I directly downloaded the cronet code:
https://github.com/hanpfei/chromium-net
Then, I identified several functions that I thought would be related based on the code and debugged them with IDA:
nativeCreateRequestAdapter
Finally, I found this point:
At this point, it was associated with libmetasec_ml.so.
Knowing the calling point, I wanted to directly call libmetasec_ml.so for easier debugging, so I wrote a program to load this so, but found it would throw an exception. Considering that the context environment might not be complete, I planned to load the so according to the normal process, also calling JNI_OnLoad:
void* hMetasecSo = dlopen(“/data/local/tmp/libmetasec_ml.so”, RTLD_LAZY);
typedef jint(*FUN)(JavaVM* vm, void* res);
FUN func_onload = (FUN)dlsym(hMetasecSo, “JNI_OnLoad”);
This call requires the JavaVM parameter, so I planned to load libart to call JNI_CreateJavaVM to create the JavaVM, setting the parameters based on online information:
options[0].optionString = “-Djava.class.path=.”;
vm_args.version = JNI_VERSION_1_6;
vm_args.options = options;
vm_args.ignoreUnrecognized = JNI_TRUE;
After compiling and running, it crashed directly. Checking the logs, it indicated that NoSigChain was not set.
Then I checked the Android source code, found the corresponding place, and saw that it was checking the option parameters for creating the VM. It should be that the no-sig-chain parameter was not present. I searched online but couldn’t find how to set this parameter, so I modified the code according to the code and combined with other properties to add one:
options[1].optionString = “-Xno-sig-chain”;
In the Android source code, it also skipped this part, and after compiling and running, the previous error was gone, but it still exited abnormally later. I checked and found the following information:
Since Android N (SDK >= 24), opening system private libraries through dlopen or relying on system private libraries in lib libraries will cause exceptions and may even crash the app.
Applications can call all so libraries in /vendor/etc/public.libraries.txt and /system/etc/public.libraries.txt, so writing the so that you want to be called into this file will make this library shared, and any application can find this so library.
I tried the above methods, including placing libart.so and related libraries in other user directories, but it still didn’t work. Considering that I originally wanted to restore the running environment, I thought of directly using the APK, which could save the trouble of creating the VM, so I wrote a test APK to call this so:
After compiling the apk, I imported it to the phone to run:
Didn’t find class “com.bytedance.mobsec.metasec.ml.MS”
Directly referenced Douyin and supplemented the corresponding package path:
Now it can run, but calling JNI_OnLoad throws an exception, so I prepared to debug with IDA and looked at the flowchart:
With llvm, the end of the so file records is Apple LLVM version 10.0.1 (clang-1001.0.46.3).
Following down, some jumps have been processed, and it is not easy to analyze, so I prepared to use unidbg (https://github.com/dqzg12300/unidbg_tools, which can be organized by this great god) to trace the code for analysis.
Based on the instruction stream obtained from the trace, I found that there was a situation where accessing “/proc/self/exe” returned -1. I directly modified this function to return 1, as this function was only used to obtain the e_machine field:
After that, I continued to trace, and combined with dynamic debugging, I found a place for code verification:
After processing the above, the test apk can run JNI_OnLoad normally. The next step is to actively call the encryption function for testing, directly calling libmetasec_ml.so in the JNI interface:
At this point, I saw information in a group saying that Douyin opened the web, so I went to check it out.
2. Douyin Web Request Parameter _signature Algorithm Analysis
Directly visit www.douyin.com, and see that the access parameter has an additional _signature, in this format:
&_signature=_02B4Z6wo00901qf0GiQAAIDAwkLkeQfbXMKn9B6AAMkm74.
Comparing several, I found that the first part (_02B4Z6wo00901) is a prefix, and the latter is initially judged to be in base64 format.
Directly searched online and found the following article:
Web Crawler – Today’s Headline _signature Parameter Reversal (First Round) _Jingwa Cannot Speak to the Sea Blog – CSDN Blog _bytedd_acrawler
https://blog.csdn.net/qq_39802740/article/details/104911315
Most of the encryption algorithm is in acrawler.js. I ran it with node as a reference. The implementation inside is a js virtual machine. I also found this article about the virtual machine:
StriveMario/jsvm: Write a Compiler for the “Some Sound” JavaScript Virtual Machine (github.com)
https://github.com/StriveMario/jsvm
However, the current version looks different from this, and the local debugging algorithm actually cannot be used, but it can still be learned.
In addition to the parameters mentioned in the article, the following several parameters need to be modified:
Then you can call global.byted_acrawler.init and global.byted_acrawler.sign parameters (which later found that the algorithm flow of this call is actually different from that run by the browser).
Using the browser for debugging also confirmed that there are init and sign functions:
The next step is to debug directly using node, as the format of the original code is like this:
All the logic is written in one line, which is inconvenient for debugging, so I first organized the code, such as this:
opCode = 3 & initCode; // initCode % 4
Then the >2 branch is ==3, and the long code is split.
After organizing, it is also convenient to add logs to output intermediate data.
During debugging, I sorted out the basic operations of the vm (vm_xor, vm_and, etc.), especially the string connection instructions, and could add logs to facilitate positioning.
The vm will check the running environment, including the following:
domDetect
debuggerDetect
nodeDetect
phantomDetect
webdriverDetect
incognitoDetect
hookDetect
locationDetect
The check results will participate in encryption as one of the factors.
At that time, I looked at several signature data and had a question: the same data always had different _signature. I thought there should be a variable factor, but I didn’t find this variable in the parameters submitted by https, so it was directly recorded in _signature.
I directly base64 decrypted the data excluding the prefix, similar to this:
<Buffer 7c 7e 62 a4 00 00 20 30 83 81 9d 5b 28 d0 87 e9 7c 76 e3 80 00 07 26 f8>
After comparing several, I didn’t find any obvious variable factors, so I guessed it might be a timestamp, but I didn’t find which data segment marked the time, so I started to look at the flow.
During the debugging process, I found a call to getTime, directly modified it to a fixed value, and the resulting _signature would not change:
That confirms that it is related to the timestamp, which means the server can use this to obtain the timestamp or the converted value to verify whether the _signature reported by the client is correct.
The main processing in the algorithm is the handling of parameter strings (location, user_agent, param) to obtain various hash values through operations (xor, or, and, etc.), then similar to base64 encryption to obtain the encrypted string (this does not directly obtain the complete plaintext and then base64 it for the last time):
After processing, the last two characters are additional verification characters, which are the low byte of a DWORD value obtained from the previous data:
_02B4Z6wo00f014U9W.wAAIDB4IuloLYVYYOFPV9AAIGw
_02B4Z6wo00f014U9W.wAAIDB4IuloLYVYYOFPV9AAIGwa2
I thought it was done, but comparing it with the browser access resulted in a tragedy, as the encryption result was different. So I went to debug it in the browser.
Having already organized the code, when accessing the browser, the js is downloaded directly. I modified the DNS to redirect this js to the local server:
127.0.0.1 sf1-ttcdn-tos.pstatp.com
In this way, when the browser accesses it, it also uses the organized js. With the previous debugging experience, it quickly clarified the process.
data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAADAAAAAQCAYAAABQrvyxAAACfElEQVRIS9WVO2hUURCGv20Eg1bapVC0sFNJJahFRPDRWFgoRokgQbAQLFJJdNFKFAVBNMROI1oYsBITsPDRpAgkNkmjEoggqIWNIILyywwMs3MxhevjwLLs3XPnzP+a0+L/XZuAE63U/wBwNz0bBWYAfcd1DrjYJfxrgHFgT9HLGeALUAIYAS6kl54A+lxNz3cAL7sEoCq7HdgVSCsBVMiPWrWszJ8EsBK4BjwEDgInrafRbKEIQJa5BawDViVrzQKHgAUrJOsdM3ZkrdP2n1sy203sPQC2eCOAW6NiX3U2JMuWCuTCKlYpIADDwBQgaV+ERpSXPgN/BLgHXAqgnCTlRxZ0dp+b7zMANb/TAPakbHQoMARMAB8BZ6/JQsqF9uw1dp7ayfJpL/AZeAR8AK4HABGwKxObVEC1HKhAVsOiQwFnQv66b0UO2/RRM2I8B1o5mA/M6aAbwHlgH3AWWJsAqLQOj6AyAA0TAdVzkVmtDgDR/xnAehtpss5kAKPfYqxtJ+hQHX4T2GzM5Wa11fd5g5UC3nRla7f2eA6xj1GpIB/rWzK/thDHO0FqSBkBup0AvAXumMcjgEWbJqrrFnS7us/dQhVQB+VuWaoAyL+PzQIRgCykMfbebKUGtgZl/EAFOjYXAWjPfmDalPqVAiJU2aruG9VtRwAxA1EBsf4MGATehFt5DvgE9AeDekDj2GxSQMNBt61Wk4VUT5mKI1v7SwWUAdlmm7HsFrpsDOgu0Cx2O30DTgFjDSH7XY+bMvDzIs0WWs6hq80i8r6HdznvdWVPBWAF8LXhNEl9xTLw15tXjxWAA8Bxu4DeAd+BjcBu4JVNEYX8n1g/AAC2uh6gEsDjAAAAAElFTkSuQmCC
Increased participation of this image data, and finally organized testing:
Overall, although the web uses a JS virtual machine, it is still weaker than the binary VM. After setting up the debugging environment, the execution process became clearer.
The web signature is basically completed here, and I will continue with the app analysis.
3. Continue APP Algorithm Analysis
I was interrupted by the last branch, and my train of thought was broken, so I familiarized myself again and continued to analyze this encryption algorithm.
After calling the encryption function of libmetasec_ml.so in the test apk, the return was NULL. After debugging, I found that it would call Java from native:
Then I referred to Douyin to complete the required packages, which contained hot update protection related code, and shielded it so that the test project could run:
//import com.bytedance.JProtect;//import com.bytedance.covode.number.Covode;//import com.meituan.robust.ChangeQuickRedirect;//import com.meituan.robust.PatchProxy;//import com.meituan.robust.PatchProxyResult;
In any case, it is various patches to simulate the previous calling environment.
I saw some strings related to root detection:
By analyzing the trace logs and filtering out some jump calculation processes, I found the possible key calls. When running with unidbg, it returned null:
The next step is to debug Douyin for verification, modifying the corresponding instruction to a loop point and setting a breakpoint at the loop:
So it was determined that the signature string returned by the following function:
For my APK call, I can directly set the start and end addresses:
//Modify the start address of so
*(unsigned int *)(dwContextAddr+0x10)=0x100000;
//Modify the end address
*(unsigned int *)(dwContextAddr+0x10)=0xFFFFFFFF;
Checking the trace code, I found that there was a check on the code instructions:
Based on the debugging process, I organized the call chain and completed the initialization call:
The test app can now directly produce results:
Now my own program can run, and the next step is to use the signing service provided in the app or work out the algorithm, which is much more convenient.
After completing the test:
Overall, the difficulty of the process analysis is not as high as wegame.
Summary of skills learned during this analysis process:
1. Develop a test APK to simulate the target calling environment
2. Unidbg simulates calling any address in so
3. RSA signature verification process, other work involved, previously only used APIs (
//1. Calculate sha256 of plaintext
//2. RSA decrypt ciphertext (generally byte stream after base64 decryption)
//3. Compare the trailing string with 1 (because the result after decryption contains this hash string, it is not equal)
Kxue ID: xwtwho
https://bbs.pediy.com/user-home-44250.htm
*This article is originally from Kxue Forum xwtwho, please indicate the source from Kxue Community when reprinting
# Previous Recommendations
1. Excellent Project Recreation: Tracker Disguised as a Thermometer
2. Further Exploration of Format String Vulnerability: CVE-2012-3569 ovftool.exe
3. Android APP Vulnerability Battle – Detailed Explanation of Content Provider Vulnerability
4. Analysis of DingTalk Invitation to Stage Function
5. Android APP Vulnerability Battle – Detailed Exploration of Activity Vulnerability
6. Basics of PHP Deserialization Vulnerability
Official Account ID: ikanxue
Official Weibo: Kxue Security
