This article is from IAM42 authorization published by IAM42‘s JianShu address:http://www.jianshu.com/u/c6f7cfa366d9

Overview

In Android development, the topics of pluginization and hotfixing are increasingly mentioned. As technology iterates and various frameworks develop, the frameworks for pluginization and hotfixing seem to have matured, and many developers are applying these two technologies in actual development collaborations and formal products. Therefore, we must understand these two technologies.

Pluginization and Hotfixing

First, it is important to clarify that pluginization and hotfixing are not the same concept. Although from a technical implementation perspective, they both start from the system loader’s point of view, whether using hook methods, proxy methods, or other low-level implementations, they all use a method of “deceiving” the Android system to allow the host to load and run the content in the plugin (patch) normally. However, their starting points are different. Pluginization, as the name suggests, aims to extract modules or functions that need to be implemented as independent components, reducing the scale of the host, and only loading the corresponding module when the relevant functionality is needed. Hotfixing, on the other hand, often starts from the perspective of fixing bugs, emphasizing the repair of known bugs without needing to reinstall the application.

For convenience of discussion, the following naming conventions will be used:

• Host: the currently running APP

• Plugin: the apk class files that need to be loaded and run in relation to pluginization technology

• Patch: the files that need to be loaded and run in relation to hotfixing technology, such as .patch, .dex, *.apk, etc., which contain dex repair content.

The hosts and plugins (patches) mentioned below all refer to the meanings above and will not be repeated.

Typical Applications of Android Pluginization Technology

The above diagram reflects the relationship between Android pluginization and hotfixing. As far as I know, in some development teams, the technology of hotfixing is used as a function for deploying daily activities on the APP side. Although there are no issues in actual results, its long-term use is still worth discussing.

Many early applications’ dynamic skin-changing functions referenced Android pluginization technology. The earliest night mode of Sina Weibo was achieved by downloading a night mode apk file. As a developer at that time, I felt it was very advanced. There are many expandable ideas for dynamically loaded applications, such as promotional activities for specific holidays, dynamic ad loading to evade review mechanisms, all of which are implementations that can be considered with Android pluginization technology. More content can be referred to in the simple and easy-to-understand introduction to Android dynamic loading technology.

Next, I will gradually describe the development history and current status of both technologies, understanding how far the hotfix framework has developed today, and generally sorting out the principles of hotfixing to gain an understanding of the existing frameworks.

Development History and Current Status of Pluginization

The origin of pluginization technology can be traced back 5 years:

• In 2012, AndroidDynamicLoader was introduced, which dynamically loads different Fragments to achieve UI replacement. It can be said to be the pioneer, but this solution has limited scalability.

• Later, 23Code appeared, which could directly download a demo of a custom control and run it.

• 2014 was a milestone year when Ren Yugang (commonly known as the chairman) released dynamic-load-apk, also known as DL. This framework provided two very important ideas:

• 1. How to manage the lifecycle of Activities within plugins: Using DLProxyActivity to call the lifecycle methods of Activities in plugins via static proxy.

• 2. How to load resource files within plugins: By reflecting on the addAssetPath method in AssetManager, specific path resources can be loaded into system memory for use.

• These two points are very meaningful, especially the second point regarding the recording of plugin resources, which has become a reference idea for many frameworks that appeared later. This framework also has some limitations; it does not support components like Service and BroadcastReceiver that need to be registered to use within plugins, and plugin apks must be implemented according to its development specifications. Overall, it still has certain costs, but it is a very valuable framework. (By the way, this framework seems to no longer be maintained; the last code update was two years ago, o(╥﹏╥)o).

• In 2015, DroidPlugin was introduced by Andy Zhang, implementing a new plugin mechanism on the Android system: it can run APK files without installation or modification, which is beneficial for improving the architecture of large apps and achieving multi-team collaborative development. This is the introduction in the DroidPlugin GitHub README. This pluginization framework comes from 360.

• In 2015, DynamicAPK was introduced, but it seems to have completely stopped updating due to licensing issues.

• In 2017, RePlugin was released, which is a pluginization framework open-sourced by 360. According to its own statements, compared to other frameworks, it only hooks the system in one place, which is ClassLoader, theoretically suggesting better stability.

• In 2017, Atlas was newly open-sourced by Alibaba, which can be said to be very powerful; for specific principles, refer to the detailed explanation of the Atlas framework; I have not used it yet.

• Finally, Small: I personally feel that Small provides a concept that is a higher level than pluginization, which is componentization, viewing a complete APP as composed of many reusable module components (this is somewhat similar to the development philosophy of React Native); developing feels like building blocks. Those interested can visit the Small official website for more information.

Hotfixing

Compared to pluginization, the use of hotfixing technology is more frequent because this technology directly addresses the products we develop, enabling faster and more convenient fixes for online bugs, which can lead to a better user experience. Therefore, I will now discuss the principles of hotfixing and explore its usage and current development status.

All the following analyses are derived from articles related to hotfixing, and here I am just organizing the conclusions. Specific analyses will not be repeated; those interested in implementation details can refer to the corresponding links. Class Loading Principles

When it comes to hotfixing, one cannot help but mention the class loading mechanism. Similar to the conventional JVM, class loading in Android is also completed through ClassLoader, specifically PathClassLoader and DexClassLoader, which are two Android-specific class loaders. The differences between these two classes are as follows:

PathClassLoader: can only load apk files already installed in the Android system (/data/app directory), and is the default class loader used in Android. DexClassLoader: can load dex/jar/apk/zip files from any directory, which refers to the patches we mentioned earlier. Both of these classes inherit from BaseDexClassLoader, and we can look at the constructor of BaseDexClassLoader.

    public BaseDexClassLoader(String dexPath, File optimizedDirectory,
            String libraryPath, ClassLoader parent) {
        super(parent);
        this.pathList = new DexPathList(this, dexPath, libraryPath, optimizedDirectory);
    }

This constructor does only one thing: it initializes a DexPathList object with the relevant parameters passed in. The constructor of DexPathList encapsulates the program files (the patch files) passed in as Element objects and adds these objects to an array collection of dexElements.

The ClassLoader loading mechanism is a particularly clever method, known as the parent delegation mechanism, under which a Class will only be loaded once. For an analysis of the ClassLoader loading mechanism and the parent delegation mechanism, refer to Android’s analysis of ClassLoader (1) Java’s ClassLoader.

It is important to understand that for a ClassLoader, loading a specific class into memory is actually done by the virtual machine. For developers, our focus should be on how to find the class that needs to be loaded.

Assuming we want to find a class named name, DexClassLoader will implement it through the following steps:

In the DexClassLoader’s findClass method, it will use a DexPathList object to call the findClass() method. In the findClass method of DexPathList, it traverses the previously constructed dexElements array collection. Once it finds a class with the same name, it directly returns that class; if not found, it returns null. In summary, finding a class through DexClassLoader is ultimately an operation of searching for a specific value in an array.

Based on all the points above, we can easily think of a very simple and crude hotfix solution. Suppose there is a bug in one or several classes in the code, after fixing the bug, we can package these classes into a patch file, then encapsulate this patch file into an Element object, and insert this Element object into the front of the existing dexElements array. This way, when DexClassLoader loads classes, it will prioritize finding the corresponding class from the inserted Element, even though the buggy class still exists in the later elements of the array, due to the characteristics of the parent delegation mechanism, this buggy class will no longer have the opportunity to be loaded, thus achieving a bug fix without reinstalling the application.

With the above idea, we could implement a simple hotfix framework ourselves. Here I recommend an article, “Hotfix – An In-Depth Principle and Implementation”, where the author deeply analyzes the principles of hotfixing and implements a basic hotfix framework based on the above principles. The implementation process is analyzed in great detail and is very suitable for understanding the principles of hotfixing.

QQ Space Super Patch Solution

After looking at the principles above, do you feel that hotfixing is simple and not worth researching? Not at all. Java is an object-oriented language, and the classes we use will have inheritance relationships and dependencies on each other. At the same time, the Android virtual machine is different from the conventional JVM; it loads not .class files but dex (specifically optimized odex files). In this process, there will inevitably be new issues worth our attention. This issue is CLASS_ISPREVERIFIED, what does it mean?

When an apk is installed, the system optimizes the dex file into an odex file, and this optimization process involves a pre-verification process. If a class’s static methods, private methods, overridden methods, and constructors reference other classes, and these classes all belong to the same dex file, then this class will be marked CLASS_ISPREVERIFIED. If, at runtime, a class marked CLASS_ISPREVERIFIED references classes from other dex files, an error will occur. Normal splitting schemes ensure that related classes are packed into the same dex file. To ensure that patches can be loaded normally, it is essential to ensure that classes are not marked CLASS_ISPREVERIFIED. To achieve this, it is necessary to implant references to classes in other dex files in the classes after splitting. The above content is excerpted from the analysis of Android hotfix technology – QQ Space Patch Solution (2).

To implant references to other classes in already compiled classes, bytecode manipulation is required. Common tools include javaassist and asm.

The QQ Space patch solution uses the bytecode manipulation method of javaassist to solve the CLASS_ISPREVERIFIED problem.

Tinker

QQ Space Super Patch, “Super Patch” often means that the patch file is large, and loading such a large folder into memory to build an Element object and insert it into the front of the array takes time, which undoubtedly affects the application’s startup speed. Therefore, Tinker proposed another idea.

Tinker’s idea is to compare the repaired class.dex with the original class.dex to generate a differential patch file patch.dex. On the phone, this patch.dex will merge with the original class.dex to generate a new file fix_class.dex, fundamentally eliminating the bug.

The approach provided by Tinker can be said to be very novel and worth learning. The process in the above diagram seems simple, but implementing it specifically is not easy. Have you ever thought about how the two .dex files compare to generate the differential patch file patch.dex? Those interested can check out Hongxiang’s analysis of “Android Hotfix Tinker Source Code Analysis of DexDiff / DexPatch”.

Of course, it should be noted that when merging patch.dex with the original class.dex, a new process is required to complete this, and considering the scale of most applications today, multidex is already quite common, so the merging strategy and success rate between multiple dex files are all issues that need to be considered when using Tinker.

For more details on Tinker, refer to “The Evolution of WeChat Android Hotfix Practice”.

Tinker provides very complete documentation and examples, making it very friendly for developers interested in integration. However, overall, the integration process is still somewhat complex, and the intrusion into the entire project is relatively strong. Tinker is the only hotfix framework I have used, and overall it is quite good. Through integration into actual applications, I gained new insights into gradle. Those interested in gradle can take a look at Tinker’s gradle integration method.

HotFix

Although the two methods mentioned above have different strategies, they all start from the perspective of the upper ClassLoader. Due to the characteristics of ClassLoader, if you want the new patch file to take effect again, whether you are implanting or merging in advance, you need to restart the application to load the new DexPathList. This means that the bug cannot be fixed without the user being aware of it. HotFix (i.e., AndFix) provides absolute advantages in this regard.

HotFix (i.e., AndFix) offers patch security services and version management on top of AndFix, making it convenient for developers. AndFix provides a way to modify field pointers at runtime in Native, achieving method replacement, allowing immediate effect without restart, and no performance consumption on the application.

For more details on AndFix, refer to https://github.com/alibaba/AndFix. Since it operates at the Native level, it cannot handle new fields or modified methods added at the Java level. Additionally, as Android has become “Android” in China, various phone manufacturers have customized their ROMs, leading to many underlying implementation differences, causing AndFix’s compatibility to not be very good.

Sophix

Alibaba launched the industry’s first non-intrusive hotfix solution, Sophix, revolutionizing the traditional release and update process for mobile terminals!

This was the title of the article I first saw when I learned about Sophix; for technical articles, using the word “revolution” can either be clickbait or genuinely impactful.

Sophix can be said to draw on the strengths of Tinker and AndFix, both of which have defects in certain aspects. Therefore, Sophix compensates for weaknesses with strengths, achieving hotfixing from a higher level through an approach of full replacement. This seems to be a consistent pattern in the development of things, where new entities summarize the experiences and lessons of predecessors, absorb good ideas, and become better.

Regarding the principles of Sophix, I have read many articles and find this one full of valuable insights, “Introduction and Analysis of Android Hotfix Solutions” is quite good, and those interested can take a look.

In summary, Sophix should be the most mature hotfix solution currently available.

Others and Conclusion

Of course, regarding the implementation of hotfixing, major companies have their own implementations, such as Ele.me’s Amigo and Meituan’s Robust, each with its advantages and disadvantages. But overall, they can be categorized into two major types:

• ClassLoader loading schemes

• Native layer replacement schemes

Alternatively, one can implement overall incremental updates to the code, referencing the approach of Android Studio Instant Run. However, this will inevitably impact performance.

In summary, it is quite challenging to find a perfect solution for hotfixing. In Android development, the four major components must be declared in advance in the AndroidManifest, and if hotfixing is needed, whether pre-occupying or dynamically modifying, it will bring strong intrusiveness (hence, Sophix does not support repairing the four major components, which is an unavoidable issue in its non-intrusive design philosophy; I wonder if there will be new methods in the future). Furthermore, the fragmentation issue of Android poses a challenge for adapting hotfix solutions. By examining several major open-source hotfix solutions on GitHub, one can see that compatibility issues are still the most frequently mentioned problems in the issues section.

Therefore, in actual development, choosing to use or which solution to choose must fit the actual application scenario. In a word, there is no best, only what is suitable.

That concludes the knowledge sorting on pluginization and hotfixing.