(Click the blue text above to quickly follow us)

For Android developers, as the project continues to grow, the compilation time for Android projects gradually increases. Sometimes, even adding a single line of code requires a long wait to see the expected results. Previously, there were relatively few tools to speed up Android compilation, among which the most representative open-source projects are Facebook’s Buck and mmin18’s LayoutCast, in addition to JRebel and Jimulabs. However, a few days ago Google announced the launch of Instant Run to speed up Android compilation, which is believed to be a blow to other tools, and this is the motivation for writing this article.

Compared to Buck, LayoutCast is lighter and has less invasive impact on the project. In August of this year, it took about a week to complete the adaptation of the company’s code. For some heavy projects, the benefits brought by Buck are obvious, but there are also many pitfalls during the adaptation process. Instant Run also has a considerable invasion of the project, but these do not require users to operate or configure, so it appears to be lightweight like LayoutCast.

Where Did the Time Go?

The general process of Android program compilation is shown in the figure, and detailed processes can be referred to in the tasks in gradle.

So why do we have to wait so long for each compilation? In fact, we can add TaskExecutionListener in gradle to monitor the execution time of each task in the gradle script.

class TimingsListener implements TaskExecutionListener, BuildListener {
    private Clock clock
    private timings = []
    @Override
    void beforeExecute(Task task) {
        clock = new org.gradle.util.Clock()
    }
    @Override
    void afterExecute(Task task, TaskState taskState) {
        def ms = clock.timeInMs
        timings.add([ms, task.path])
        task.project.logger.warn "${task.path} took ${ms}ms"
    }
    @Override
    void buildFinished(BuildResult result) {
        println "Task timings:"
        for (timing in timings) {
            if (timing[0] >= 50) {
                printf "%7sms  %s\n", timing
            }
        }
    }
    @Override
    void buildStarted(Gradle gradle) {}

    @Override
    void projectsEvaluated(Gradle gradle) {}

    @Override
    void projectsLoaded(Gradle gradle) {}

    @Override
    void settingsEvaluated(Settings settings) {}
}gradle.addListener new TimingsListener()

By executing the script, it can be found that the main time-consuming parts are dex (including preDex) and install. The main work of BUCK and LayoutCast is also focused on these time-consuming steps.

How to Speed Up?

During development, modifications to the project generally fall into two categories: modifications to Java files and modifications to resource files. These modifications will involve the aforementioned time-consuming steps, which is why even modifying a single line of code requires a long compilation time.

1. Modifying Java Files

Typically, modified .java files will first go through javac to generate .class files, which are then processed with other .class files to generate .dex files through dx. The operation of dx is time-consuming. To address this, BUCK, LayoutCast, and Instant Run adopt two methods.

BUCK

BUCK establishes a complete set of dependency rules and a refined caching system to reduce compilation time and uses a third-party dex merge tool to reduce the time complexity of merging .dex files from O(N^2) to O(NlgN).

As shown in the figure, when modifying A.java, it only involves the corresponding dx operation and dex merge operation (red part), thus greatly reducing the dx operation time. BUCK has made great efforts in dependency rules and introduced ABI, further reducing unnecessary operations.

LayoutCast

LayoutCast’s implementation is similar to the principles of many plugins. The specific analysis is as follows:

When the ClassLoader looks for a class, it first calls the findClass method in the BaseDexClassLoader class.

//----dalvik/system/BaseDexClassLoader.java  
 protected Class<?> findClass(String name) throws ClassNotFoundException {
        Class clazz = pathList.findClass(name);
        if (clazz == null) {
            throw new ClassNotFoundException(name);
        }
        return clazz;
    }

Then it looks for the corresponding class in the DexPathList class based on dexElements.

//----dalvik/system/DexPathList.java  
public Class findClass(String name) {
        for (Element element : dexElements) {
            DexFile dex = element.dexFile;
            if (dex != null) {
              Class clazz = dex.loadClassBinaryName(name, definingContext);
              if (clazz != null) {
                  return clazz;
              }
            }
        }
        return null;
    }

Here, dexElements represents different dex files.

/** list of dex/resource (class path) elements */
    private final Element[] dexElements;

This means that when the ClassLoader loads a class, it will look for it in the order of dex files in dexElements. As shown in the figure, if class A is found in 1.dex, it will not continue to search in subsequent dex files.

LayoutCast utilizes this principle to generate dex files from modified Java files and inserts this dex file at the front of the dexElements array using reflection. Of course, the process from Java to dex requires additional work to look up various dependency packages, which is implemented in cast.py.

This implementation works fine under ART, but will encounter IllegalAccessError issues under Dalvik.

java.lang.IllegalAccessError: Class ref in pre-verified class resolved to
             unexpected implementation
dalvik.system.DexFile.defineClass(Native Method)
dalvik.system.DexFile.loadClassBinaryName(DexFile.java:211)
dalvik.system.DexPathList.findClass(DexPathList.java:315)
dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.j

The specific reasons and solutions can refer to Bugly’s article.

Install Run

Install Run also generates new incremental dex, but the classes in the new dex have different names from the original classes. For example, after modifying Hello.java, a dex file containing the Hello$override class will be generated.

So how is the Hello$Override class in the new dex file called?

First, let’s look at the difference before and after the compilation of the original Hello.java file:

Before compilation of hello.java:

public String name(String str) {
	return str;
}

After Instant Run compilation:

---compiled  Hello.java
public String name(String str) {
	IncrementalChange var2 = $change;
	return var2 != null?(String)var2.access$dispatch(
		"name.(Ljava/lang/String;)Ljava/lang/String;", 
		new Object[]{this, str}):str;
}

It can be seen that if $change exists, the corresponding function in $change will be called. So we just need to use reflection to change the $change field in Hello.java to the modified Hello$override class.

This is why Instant Run does not encounter the IllegalAccessError issue mentioned earlier and supports seeing the modification effect without restarting. For more details, you can check the blog of Hanjiang Bu Diao.

2. Modifying Resources

Modifications to resource files involve AAPT, ApkBuilder, and the final Install operation. The operation of AAPT requires higher specifications, and neither LayoutCast nor Instant Run has optimized this part; their main work is in the latter two operations. The main idea is to package the modified resources into a new .ap_ file using aapt and use reflection to change the original resource files to the modified ones.

LayoutCast

LayoutCast does two main things.

Modifying the LayoutInflater service

We are familiar with the following usage:

LayoutInflater layoutInflater = LayoutInflater.from(context);
View view = layoutInflater.inflate(resourceId, root);

Where LayoutInflater.from is implemented in the Context implementation class ContextImp to obtain the LAYOUT_INFLATER_SERVICE system service.

//----  android/view/LayoutInflater.java
public static LayoutInflater from(Context context) {
         LayoutInflater LayoutInflater =
                 (LayoutInflater)context.getSystemService(Context.
                 LAYOUT_INFLATER_SERVICE);
         if (LayoutInflater == null) {
             throw new AssertionError("LayoutInflater not found.");
         }
         return LayoutInflater;
     }

So how does ContextImpl obtain the corresponding service? Looking at the ContextImpl class, we can find:

//---- android/app/ContextImpl.java
public Object getSystemService(String name) {
        ServiceFetcher fetcher = SYSTEM_SERVICE_MAP.get(name);
        return fetcher == null ? null : fetcher.getService(this);
    }

It can be seen that the process of calling getSystemService is to look up in the SYSTEM_SERVICE_MAP table for the ServiceFetcher and return the mCachedInstance in ServiceFetcher. So we only need to replace mCachedInstance with our custom BootInflater and complete the Resource override in BootInflater, as shown in the figure below.

Modifying Resources

We know that Activity accesses resources by calling getResources(), which actually calls the getResource() method in the ContextWrapper class.

public Resources getResources(){
         return mBase.getResources();
}

LayoutCast adopts the method of replacing mBase with a custom OverrideContext and returning the modified Resource.

Instant Run

Instant Run’s handling of resource files is basically similar to LayoutCast, but there are differences in detail handling. For example, Instant Run modifies the ActivityThread class’s mPackages and mResourcePackages to change the value of mResDir in LoadedApk.

for (String fieldName : new String[] { "mPackages", "mResourcePackages" })
{
  Field field = activityThread.getDeclaredField(fieldName);
  field.setAccessible(true);
  Object value = field.get(currentActivityThread);
  for (Map.Entry<String, WeakReference<?>> entry : ((Map)value).entrySet())
  {
    Object loadedApk = ((WeakReference)entry.getValue()).get();
    if (loadedApk != null) {
      if (mApplication.get(loadedApk) == bootstrap)
      {
        if (externalResourceFile != null) {
          mResDir.set(loadedApk, externalResourceFile);
        }
        if ((realApplication != null) && (mLoadedApk != null)) {
          mLoadedApk.set(realApplication, loadedApk);
        }
      }
    }
  }
}

Resource file modification is more complex than Java file modification, involving issues such as aapt, attribute uniqueness, and ID consistency, which increases the difficulty of resource file handling.

Conclusion

In summary, each method has its own characteristics. BUCK relies on its powerful caching and dependency management system, while LayoutCast and Instant Run adopt more flexible methods. Relatively speaking, Instant Run, with its inherent advantages (and combination with the upgraded gradle), can outperform LayoutCast, but the idea of LayoutCast is still commendable. Currently, incremental compilation is concentrated on modifications to Java files, and it seems that modifications to resource files are not yet supported; improvements in this area should be expected in the future.

If you like this article, remember to like and share it with your friends, and everyone is welcome to contribute articles.

Welcome to follow us and discuss technology together. Scan and long press the QR code below to quickly follow us

Or search for the WeChat public account: JANiubility, to follow us.

Click the “Join the group to learn” menu in the lower right corner to join the group for learning and discussion!