At the beginning of this year, we found that after the Sync phase in Android Studio, there was a long waiting time. The following image shows the sync duration from the second half of 2021, which is about 10 minutes.

We eventually pinpointed that the long wait was caused by the Jetifier operation during the sync process, which mainly replaces Support dependencies with AndroidX dependencies. Regardless of whether the project contains support dependencies, if enableJetifier = true is turned on, the conversion will take place. Recently, the Android team also released a warning about Jetifier being enabled in the latest Chipmunk patch 1 version Jetifier Enabled Warning(https://developer.android.com/studio/releases/gradle-plugin?buildsystem=ndk-build#jetifier-build-analyzer). Earlier, tools like gradle-doctor (https://runningcode.github.io/gradle-doctor/) also issued warnings regarding Jetifier being enabled.

Jetifier was enabled which means your builds are slower by 4-20%.

Ultimately, we replaced all Support dependencies within the project with AndroidX dependencies, reducing our sync time by 2 minutes and 30 seconds.

Incremental build time was reduced by 28%

Analyzing from the Gradle lifecycle, Gradle is divided into three phases: initialization, configuration, and execution.

• The initialization phase allocates a Gradle instance, executes the init.gradle script, and generates the Setting object to determine which modules will participate in the build process.

• The configuration phase accesses the build.gradle of all participating modules to obtain the DAG (Directed Acyclic Graph) for this task execution.

• The execution phase executes the tasks on the TaskGraph based on the DAG determined in the configuration phase.

When analyzing the sync process, we found that there were actually no tasks being executed, and the Sync process was more like executing an empty task but with some strange parameters. A rough simulation using the command line looks like this, but note that this command is a rough simulation because it does not return a model to the IDE for project indexing, and the IDE does not provide external command invocation.

 ./gradlew -P android.injected.build.model.only.advanced=true -P android.injected.build.model.only=true -P android.injected.build.model.disable.src.download=true -P org.gradle.kotlin.dsl.provider.cid=465291211918416 -P kotlin.mpp.enableIntransitiveMetadataConfiguration=true -P android.injected.invoked.from.ide=true -P android.injected.build.model.only.versioned=3 -P android.injected.studio.version=2022.2.1 -P idea.gradle.do.not.build.tasks=false -D idea.active=true -D idea.version=2022.2 -D java.awt.headless=true -D idea.sync.active=true -D org.gradle.internal.GradleProjectBuilderOptions=omit_all_tasks --init-script /private/var/folders/0c/zkqf7nrj6pl_d40jp4m9h3xm0000gn/T/sync.studio.tooling11.gradle  --init-script /private/var/folders/0c/zkqf7nrj6pl_d40jp4m9h3xm0000gn/T/ijmapper.gradle --init-script /private/var/folders/0c/zkqf7nrj6pl_d40jp4m9h3xm0000gn/T/ijinit.gradle

Having confirmed that no tasks were executed during the sync phase, we focused our analysis on the configuration phase process. After repeatedly observing the log output, we found that there were post outputs after the sync ended, with the term JetifyTransform appearing frequently.

> Transform jetified-openDefault-10.10.0.aar (project :weibo) with JetifyTransformWARNING: [XmlResourcesTransformer] No mapping for: android/support/FILE_PROVIDER_PATHSWARNING: [XmlResourcesTransformer] No mapping for: android/support/FILE_PROVIDER_PATHS

It was inferred that the Jetifier operation caused product conversion during the sync phase, and setting android.enableJetifier=false made the JetifyTransform post output disappear again. Thus, we identified the root cause as the Jetifier operation extending the overall sync duration.

Jetifier is implemented through Gradle’s Artifact transform (https://docs.gradle.org/current/userguide/artifact_transforms.html#implementing_incremental_artifact_transforms), which mainly converts dependency products. Unlike AGP’s local Transform for intermediate products, Gradle’s global product conversion ensures that the transformed products can be used after sync, allowing for simultaneous handling of class, layout, and manifest support dependencies in a single transform process, which is not achievable in AGP’s transform.

@CacheableTransformabstract class JetifyTransform : TransformAction<JetifyTransform.Parameters> {}

The input for Jetify Transform is aar/jar, which scans the files within the aar one by one, performing the corresponding file type transformations and writing them back. The transformations for Class and Xml files can be divided into:

• Class files: Bytecode transformation using ASM operations based on mapping.

• Xml files: Parsing XML files using regular expressions and performing string transformations based on mapping.

After the transform ends, the extracted files need to be compressed again into jeitfier-xxx.aar/jar for downstream transforms to consume.

We know the problem was caused by Jetifier, so theoretically, we could hook the AGP to read results by modifying the value of android.enableJetifier in memory before the sync phase. My colleagues and I began trying to modify the Gradle configuration at runtime.

First, we tried adding android.enableJetifier=false and android.useAndroidX=false parameters to gradle.startParameter.projectProperties or gradle.startParameter.systemPropertiesArgs. These two configurations are global parameters for gradle. However, we found that the modifications did not take effect, and the JetifierTransform log still appeared.

Next, we tried modifying the value of android.enableJetifier=false in settings.ext, but the modification still did not take effect.

After reviewing the source code, we found that this value is read by AGP through BasePlugin#ProjectServices, which is a common base class for com.android.library and com.android.application plugins. As long as we successfully hook the reading value here, we can turn off Jetifier transform.

// com.android.build.gradle.internal.DependencyConfigurator#configureDependencyChecksfun configureDependencyChecks(): DependencyConfigurator {        val useAndroidX = projectServices.projectOptions.get(BooleanOption.USE_ANDROID_X)        val enableJetifier = projectServices.projectOptions.get(BooleanOption.ENABLE_JETIFIER)}

class TurnOffJetifierPlugin : Plugin<Project> {        override fun apply(project: Project) {               project.plugins.withType(BasePlugin::class.java) {                   val service = it.getProjectService() ?: return@withType                   val service = it.getProjectService() ?: return@withType                   val projectOptions = service.projectOptions                   val projectOptionsReflect = Reflect.on(projectOptions)                   val optionValueReflect = Reflect.onClass(                       "com.android.build.gradle.options.ProjectOptions$OptionValue",                        projectOptions.javaClass.classLoader                   )                    val defaultProvider = DefaultProvider() { false }                   val optionValueObj = optionValueReflect.create(projectOptions, BooleanOption.ENABLE_JETIFIER).get<Any>()                   Reflect.on(optionValueObj)                             .set("valueForUseAtConfiguration", defaultProvider)                             .set("valueForUseAtExecution", defaultProvider)                   val map = getNewMap(projectOptionsReflect, optionValueObj)                   projectOptionsReflect.set("booleanOptionValues", map)             }        }                private fun BasePlugin<*, *, *>?.getProjectService() =                Reflect.on(this)                       .field("projectServices")                       .get<ProjectServices?>()}

In the demo test, we found it to be feasible, and then we planned to try to apply this plugin in all modules:

// Root project apply pluginallProjects {        apply plugin: TurnOffJetifierPlugin.class} // Include builded project apply plugin: settings.gradle.startParameter.initScripts = [initFile] # initFile allprojects { Project project ->      apply plugin: TurnOffJetifierPlugin.class}

We found that applying the plugin in the root project successfully modified the value, but it did not succeed for the included built repositories. In the previous article 《Bilibili Android Build Optimization》, it was mentioned that our repositories are organized through Composite builds, and triggering the initialization of the included build sub-repositories is achieved through Init Script. The apply plugin in the Init Script occurs at the same time as afterEvaluate, and the modification occurs later than the reading timing, thus the modification still failed.

We considered that even if we found the right timing to modify the value, such a black magic relies on the corresponding AGP implementation, increasing maintenance costs. Moreover, turning off Jetifier operations during the sync phase does not solve the transform during the build and packaging phase, which is only a temporary solution. Our true demand is to remove all support packages, ultimately completely shutting down Jetifier operations in all phases. Therefore, we decided to migrate the remaining support dependencies.

Thanks to the MonoRepo architecture, the source code within the project has already replaced Support dependencies with AndroidX dependencies. By removing support dependencies from the configuration phase and enabling A8 checks, we can identify the remaining support dependencies in the repository. For information on what A8 checks are, please refer to the subsequent anti-degradation chapter. The legacy code includes a few internal libraries that are outside the large repository and some external third-party libraries. For these repositories, we have taken the following three migration measures:

Internal Libraries

For internal libraries, we notified the codebase owners to migrate, performing the operation as AS → Refactor → Migrate to AndroidX, and then manually publishing to the remote Maven, updating the version number of the internal library in the main repository.

Third-party Libraries with Source Code Access

After cloning the third-party library source code, if the other party directly supports AndroidX and the Change Log indicates no breaking changes, we directly release the upgraded version of the third-party library; in other cases, we choose to publish the library upgraded through the automated migration operation under the same version number to our company’s Nexus repository. Finally, we update the main repository version number.

Third-party Libraries without Source Code Access (Only AAR and POM)

We packaged some common CLI tools as babeltools, which internally integrated Jetifier CLI(https://developer.android.com/studio/command-line/jetifier). By executing the following command, we can convert AAR/JAR containing support to AndroidX AAR/JAR:

bJetifier -i [input.aar] -o [output.aar]

If there is a POM file, it needs to be manually modified to change the corresponding support dependencies to AndroidX. Reference for corresponding dependency conversion can be found in CSV (https://developer.android.com/topic/libraries/support-library/downloads/androidx-class-mapping.csv).

Some Pitfalls Encountered During Migration

Package Size Changes

After completing the migration of support packages, we compared the package sizes using Android Studio, finding a size reduction of 6KB on 32-bit and 600KB on 64-bit. Theoretically, the size difference at the same commit point after migration should not exceed 1KB.

Using the diffuse (https://github.com/JakeWharton/diffuse) tool to compare the differences between APKs, we found that the primary impact points on package size came from two aspects:

• R.class

• Size differences in .so files

The reduction in R.class was mainly due to an internal library being upgraded to AndroidX, causing R file transmission issues. The solution was to enable non-transitive R for that module.

The size differences in .so files arose from inconsistencies between the NDK versions used for our published Fresco and the official release. We resolved this by replacing the .so files generated during packaging with those from the previous AAR, thereby reducing the size difference.

After fixing the above issues, we compared the same commit points again, and the size difference was reduced to within 1KB.

Nexus Upload

For some third-party libraries that only have jar and aar files, such as com.tencent.tauth:qqopensdk and com.huawei.android.hms:security-base, they need to be manually uploaded to Nexus. The manual upload page UI looks like this:

When uploading an AAR with a suffix like -bili4 to Nexus, it will automatically append the suffix as a classifier, causing com.facebook.fresco:imagepipeline:1.13.0-bili4 to not match correctly. Its corresponding dependency is com.facebook.fresco:imagepipeline:1.13.0:bili4.

Additionally, for com.github.bmelnychuk:atv:1.2.9, which includes a POM dependency, it is necessary to upload the POM file along with it. The POM describes the module’s dependency information, and losing the POM file will cause compilation errors due to missing dependencies.

AndroidX Log Checks

After turning off Jetifier operations, the Jetify Transform output log disappeared, but there were still many AndroidX check logs in the post-sync phase, as shown below, with only one data point displayed.

WARNING: Your project has set `android.useAndroidX=true`, but jetified-openDefault-10.10.0.aar still contains legacy support libraries, which may cause runtime issues. This behavior will not be allowed in Android Gradle plugin 8.0. Please use only AndroidX dependencies or set `android.enableJetifier=true` in the `gradle.properties` file to migrate your project to AndroidX (see https://developer.android.com/jetpack/androidx/migrate for more info).

However, using jadx to unpack revealed that the AAR no longer contained support dependencies. This check is not accurate.

After reviewing the source code, we found that when enableJetifier is turned off and useAndroidX is turned on, AGP will enable AndroidX checks for developers, which consumes time proportional to the project size.

    class AndroidXEnabledJetifierDisabled(            private val project: Project,            private val configurationName: String,            private val issueReporter: IssueReporter    ) : Action<ResolvableDependencies> {        private val issueReported =                "${AndroidXEnabledJetifierDisabled::class.java.name}_issue_reported"        override fun execute(resolvableDependencies: ResolvableDependencies) {            // Report only once            if (project.extensions.extraProperties.has(issueReported)) {                return            }        } }

To prevent duplicate checks, a switch was added in the source code. We can close the AndroidX check by adding the following parameter in the gradle.properties file.

com.android.build.gradle.internal.dependency.AndroidXDependencyCheck$AndroidXEnabledJetifierDisabled_issue_reported=true

To ensure that future business parties integrating third-party SDKs do not reintroduce support dependencies through transparent dependency methods, we will exclude all support dependencies during the configuration phase. This guarantees that the APK produced will no longer contain support dependencies. However, if the APK at runtime uses these dependencies, the app will crash. Our solution is to use the A8 checks mentioned earlier in the article 《Bilibili Android Build Optimization》 to check for missing dependencies during the pipeline merge, thereby preventing the introduction of new support dependencies. The code to exclude support dependencies is as follows:

allprojects {    configurations.all { Configuration c ->        if (c.state == Configuration.State.UNRESOLVED) {                exclude group: 'com.android.support'                exclude group: 'android.arch.core'                exclude group: 'android.arch.lifecycle'                exclude group: 'android.arch.persistence.room'                exclude group: 'android.arch.persistence'                exclude group: 'com.squareup.leakcanary', module: "leakcanary-object-watcher-android-support-fragments"        }    }}

Since May 2018, Hello World, AndroidX(https://android-developers.googleblog.com/2018/05/hello-world-androidx.html), AndroidX has entered the vision of Android development. With the release of 28.0 version of the support package(https://developer.android.com/topic/libraries/support-library/revisions?hl=zh-cn#28-0-0), support packages have stopped maintenance, and all updates are iterated only on AndroidX. In a recent blog post on Migrate to AndroidX (https://developer.android.com/jetpack/androidx/migrate), a warning was issued:

Caution: As of late 2021, most of the library ecosystem already supports AndroidX natively. This means that your project is most likely already using AndroidX libraries directly and there is no need to follow the steps in this migration guide. Additionally, the enableJetifier (https://developer.android.com/jetpack/androidx/migrate#migrate_an_existing_project_using_android_studio) flag mentioned in this guide can lead to slower build times and should not be used unless it’s necessary.

If your project already has the enableJetifier flag and it’s turned on, you can run Build Analyzer’s Jetifier check to confirm if it’s actually needed. The Build Analyzer check is available starting in Android Studio Chipmunk (https://developer.android.com/studio/preview/features#jetifier-build-analyzer).

If your project still has a small number of transitive dependencies from Support, I believe it is necessary to migrate them. Keeping the overall toolchain updated is a high return on investment for improving compilation performance.