Boosting Your Android App's Performance: A Guide to Macrobenchmarking, Baseline Profiles, and Startup Profiling

10 min readMar 1, 2024

Let’s move to the next section on improving your apps after KTlint its time for benchmarking!

Introduction

In today’s competitive mobile landscape, delivering a seamless and performant app experience is crucial. Identifying and streamlining performance bottlenecks early in the development process is essential. This guide will help you to work with Macrobenchmarking, Baseline Profiles, and Startup Profiling — powerful techniques for comprehensively analyzing and optimizing your Android app’s performance.

Why do we need this?

Who doesn’t need this smooth and fast app-opening experience?
Smooth and responsive apps keep users engaged and coming back for more! These are the benefits which compels us to learn more about the optimization of our application.

Faster App Launches
Targeted Optimization
Consistent Performance across different devices
Early Issue Detection
Improved Memory Efficiency
Enhanced Battery Life
Increased User Engagement

Understanding Android Runtime Compilation and Virtual Machines

Android utilizes different virtual machines for app execution:

Dalvik (Up to Android 4.4 KitKat):

Interpreted bytecode, meaning code was translated line by line during execution, leading to slower performance.
Used on older devices and offered a smaller app footprint.
[Every runtime] Plain Java Code(.java) — (fed) → JIT Compiler → Bytecode(.dex->.odex) — (fed) →DVM
dexOpt is used to create an optimized version of .dex i.e. .odex.
DEX( Dalvik Executable) or ODEX ( Optimized Dalvik Executable)
DVM: Converts the Dalvik byte-code into machine code

2. ART (Android 5.0 Lollipop and above):

Ahead-of-time (AOT) compilation converts the bytecode into native machine code before installation.
Also utilizes Just-In-Time (JIT) compilation for dynamic optimization and Profile-Guided Optimization(PGO) for better performance.
Takes part of the app and does full compilation during installation.
Uses dex2oat to convert dex into .odex(.oat) an Android native code loaded in runtime. (OAT = Of Ahead Time’ = Ahead of Time)
It offers faster app launch times but increases installation size and requires more storage space.
ART: Converts the native code into machine code

“Native code is faster than Java bytecode because it is compiled directly into the machine code that the computer’s processor can understand. There is no need for an intermediate layer of interpretation, which can add some overhead.
Java bytecode is platform-independent. This means that it can be executed on any platform that has a Java Virtual Machine (JVM). It must first interpret the bytecode into machine code before it can be executed. This can make Java programs slower than native code programs.”

Android employs several runtime compilation strategies to balance performance and resource usage:

Ahead-Of-Time (AOT): Pre-compiles bytecode into native machine code before installation, offering faster app launch but with increased installation time and storage footprint. (.oat/.odex files!)
Just-In-Time (JIT): Translates bytecode into machine code at runtime, providing flexibility and smaller app sizes. However, it can lead to runtime overhead and slower startup times. (.dex and .odex files!)
Profile-Guided Optimization (PGO): (Introduced in Nougat) Analyzes execution profiles to prioritize AOT compilation for frequently used methods, further enhancing performance based on usage patterns. (part of files that need to go through AOT!)

make sure you check this out https://proandroiddev.com/android-runtime-how-dalvik-and-art-work-6e57cf1c50e5

Android Version Support and Compiler Evolution

Android 1.0: Relied on an interpreter, resulting in lower performance and higher memory usage. (DVM)
Android 2.2 (Froyo): Introduced a combination of interpreter and JIT, improving performance but still facing limitations. (DVM)
Android 5.0 (Lollipop): First ART version introduced here! Introduced Full AOT compilation, significantly improving startup times but increasing installation time and storage usage.

You must have seen this in such devices! Since the update replaces all system apk so the android will need to re-generate those baseline profiles again for your smoother experience. It created .odex files for all at the time of first boot.

Android 7.0 (Nougat): Leveraged JIT with ProfileGuidedOpt (list of files ->AOT) utilizes both odex+dex files for more efficient code optimization. Whenever the app finds out a new hot path then the whole flow runs again with new odex files with new profile.

Android 9+: Integrated Cloud Profiles over app metadata to optimize performance for new users by crowd-sourcing. However, maintaining performance across diverse devices remains a challenge.

But what if the developers have a new rollout? Since the profiles need to be crowdsourced again, this curve will be slow again!!

Then comes Baseline and Startup Profiling! where we can specify the profile on pre-build so it will smoother theis app rollout where we don't have the crowd-sourced profiling

Benchmarking and Types

Benchmarking involves measuring and evaluating an application’s performance to identify areas for improvement. There are two main types:

Macrobenchmarking: Focuses on high-level performance metrics like app startup time, responsiveness, and memory usage, simulating typical user behavior and measuring real-world performance. Make sure your Gradle plugin version is 7 or higher.
Microbenchmarking: Targets specific code segments or functions to measure their individual execution times and resource consumption, isolating code performance and pinpointing specific bottlenecks.

Baseline Profiles and Startup Profiles:

Baseline Profile: Captures essential information about the application’s codebase, aiding optimization efforts by specifying which files must be compiled Ahead Of Time (AOT) during installation for improved startup performance.
Startup Profile: Records startup performance metrics, enabling developers to identify and address bottlenecks affecting app launch times and providing information about the compilation and loading processes during startup.

Steps to Add Benchmarking and Profiles:

1. Create a Benchmark Module:

Right-click your project in Android Studio and select “New > Module.”
Choose “Benchmark” from the Templates pane and opt for MacroBenchmark
Customize the module name (e.g., “benchmark”) and click “Finish.”

Now after creation of this benchmark module lets follow this! We will first try to see changes in app module, library module and then benchmark module

App Module Changes:

Create benchmark-rules.pro:

In your app module’s directory (app/), create a new file named benchmark-rules.pro. Add the following line to prevent obfuscation of benchmarking classes during proguard processing:

-dontobfuscate

Modify :app/build.gradle.kts:

Open your app module’s build.gradle.kts file and make the following changes:

buildTypes {
    release {
        isMinifyEnabled = false
        proguardFiles(
            getDefaultProguardFile("proguard-android-optimize.txt"),
            "proguard-rules.pro"
        )
    }
    create("benchmark") {
        initWith(buildTypes.getByName("release"))
        signingConfig = signingConfigs.getByName("debug")
        matchingFallbacks += listOf("release")
        isDebuggable = true // Set to true for debuggable benchmark builds
        proguardFiles("benchmark-rules.pro") // Include the rules file
    }
}

dependencies {
    implementation "androidx.profileinstaller:profileinstaller:1.4.0-alpha01" // Optional, simplifies profile deployment
}

We create a new build type named “benchmark” that inherits from the “release” build type but disables debugging and enables proguard rules (benchmark-rules.pro).
Setting isDebuggable to true allows easier debugging of benchmark tests.
The androidx.profileinstaller dependency is optional and helps with profile deployment on devices. It should be added to every module.

Enable Profileable Flag in Your Manifest:

Add the following attribute to the <application> tag in your app's manifest file:

<profileable
            android:shell="true"
            tools:targetApi="29" />

Library Module Changes (if applicable):

Plugins and Configuration:

If you have a library module (mylibrary), ensure it includes the following plugins in its build.gradle.kts file:

plugins {
    id("com.android.library") // Library plugin
    id("org.jetbrains.kotlin.android") // Kotlin plugin
    id("androidx.baselineprofile") version "1.2.3" // Baseline profile plugin
}

Modify the android block in your library module's build.gradle.kts to include settings for the "benchmark" build type:

android {
    // ... other configurations

    buildTypes {
        release {
            isMinifyEnabled = false
            proguardFiles(
                getDefaultProguardFile("proguard-android-optimize.txt"),
                "proguard-rules.pro"
            )
        }
        create("benchmark") {
            initWith(buildTypes.getByName("release"))
            signingConfig = signingConfigs.getByName("debug")
            matchingFallbacks += listOf("release")
        }
    }

    baselineProfile {
        // Filters the generated profile rules (optional)
        // This example keeps the classes in the `com.example.mylibrary.**` package and its subpackages.
        filter {
            include("com.example.mylibrary.**")
        }
    }
}

We ensure the library module has the baseline profile plugin applied.
We configure the “benchmark” build type similarly to the app module but without modifying isDebuggable.
The optional baselineProfile block allows you to filter the generated profile rules to include only classes relevant to your library.
Add the following dependency in your library module’s build.gradle.kts:

dependencies {     
    implementation "androidx.profileinstaller:profileinstaller:1.4.0-alpha01" // Optional, simplifies profile deployment
    baselineProfile(project(":benchmark")) // Reference the benchmark module 
}

This dependency adds the benchmark module as a baseline profile dependency for your library.

Benchmark Module:

Inside the benchmark module, add a test device. We chose pixel2 with API level 34 and make sure we suppress errors for our emulator.

defaultConfig {
    minSdk = 24
    targetSdk = 34
    testInstrumentationRunnerArguments["androidx.benchmark.suppressErrors"] = "EMULATOR, DEBUGGABLE"
    testInstrumentationRunner = "androidx.test.runner.AndroidJUnitRunner"
}

testOptions.managedDevices.devices {
    create<ManagedVirtualDevice>("pixel2api34") {
        device = "Pixel 2"
        apiLevel = 34
        systemImageSource = "aosp-atd"
    }
}
targetProjectPath = ":app" // Set the target project for the benchmark

5. Create a Baseline Profile (in the benchmark module):

Create a class extending androidx.benchmark.macro.junit4.BaselineProfileRule.
Override the @Test-annotated generateBaselineProfile method to simulate user behavior and capture the baseline profile. You must have noticed thatincludeInStartupProfile and profileBlock is added, which makes sure the user wants a baseline profile along with a startup Profile or not. includeInStartupProfile helps to focus optimization efforts on the most critical parts of the startup process. The code within this profileBlockblock directly contributes to the profile information collected by the benchmarking tool.

@RunWith(AndroidJUnit4::class)
class BaselineProfileGenerator {
    @RequiresApi(Build.VERSION_CODES.P)
    @get:Rule
    val baselineRule = BaselineProfileRule()
    @RequiresApi(Build.VERSION_CODES.P)
    @Test
    fun generateBaselineProfile() = baselineRule.collect(
        packageName = "com.example.sample",
        includeInStartupProfile = true,
        profileBlock = {
            // Run your app here, and perform the actions you want
            // to capture in the baseline profile
            startActivityAndWait() // library itself captures the startup time
            device.wait(Until.hasObject(By.res("com.example.sample", "imageView")), 7000)
            // now lets scroll app
            val feed = device.findObject(By.res("com.example.sample", "imageView"))
            feed.fling(Direction.DOWN)
        },
    )
}

6. Create a Startup Benchmark (in the benchmark module):

Create a class extending androidx.benchmark.macro.junit4.MacrobenchmarkRule.
Implement the @Test-annotated startup or startup(CompilationMode mode) methods to simulate app launch and measure startup time:

@RunWith(AndroidJUnit4::class)
class ExampleStartupBenchmark {
    @get:Rule
    val benchmarkRule = MacrobenchmarkRule()
    
    @Test
    fun startup() = benchmarkRule.measureRepeated(
        packageName = "com.example.sample",
        metrics = listOf(StartupTimingMetric(), FrameTimingMetric()),
        iterations = 5,
        startupMode = StartupMode.COLD,
    ) {
        pressHome()
        startActivityAndWait()
    }
    @Test
    fun startUpCompilationModeWithoutBaseline() {
        startup(CompilationMode.None())
    }
    @Test
    fun startUpCompilationModeWithBaseline() {
        startup(
            CompilationMode.Partial(
                baselineProfileMode = BaselineProfileMode.Require,
            ),
        )
    }
    fun startup(mode: CompilationMode) = benchmarkRule.measureRepeated(
        packageName = "com.example.sample",
        metrics = listOf(StartupTimingMetric(), FrameTimingMetric()),
        iterations = 5,
        startupMode = StartupMode.COLD,
        compilationMode = mode,
    ) {
        pressHome()
        startActivityAndWait()
    }
}

Understanding Compilation Modes:

Macrobenchmarking allows you to experiment with different compilation modes to study their impact on startup performance:

None: No pre-compilation at all, simulating a fresh install and potentially worst-case behavior.
Partial: Pre-compiles some methods based on usage patterns, representing a typical user’s app experience.
Partial with Baseline Profile: Similar to Partial, but guided by the baseline profile for more accurate optimization.
Full: Full AOT compilation representing an already optimized app (often used for system app benchmarks).

7. Generate Baseline and Startup Profiles (using Gradle commands):

Navigate to Android Studio’s “Terminal” window or your command line.
Make sure you replace PACKAGE_NAME {com.example.benchmark where your baselineProfileGenerator exists}
Execute the following command:

./gradlew :benchmark:allDevicesBenchmarkAndroidTest --rerun-tasks -P android.testInstrumentationRunnerArguments.class=com.example.benchmark.BaselineProfileGenerator

Or One can create a gradle task for this in the benchmark module

tasks.register<Exec>("generateBaselineProfiles") {
    commandLine(
        ".././gradlew",
        "allDevicesBenchmarkAndroidTest",
        "--rerun-tasks",
        "-P",
        "android.testInstrumentationRunnerArguments.class=com.example.benchmark.BaselineProfileGenerator",
    )
}

tasks.register<Exec>("installBaselineProfiles") {
    dependsOn("generateBaselineProfiles")
    doLast {
      commandLine(
        "cp",
        "-R",
        "$rootDir/benchmark/build/outputs/managed_device_android_test_additional_output/benchmark/pixel2api34/BaselineProfileGenerator_generateBaselineProfile-baseline-prof.txt",
        "$rootDir/app/src/main/baseline-prof.txt"
      )
      commandLine(
          "cp",
          "-R",
          "$rootDir/benchmark/build/outputs/managed_device_android_test_additional_output/benchmark/pixel2api34/BaselineProfileGenerator_generateBaselineProfile-startup-prof.txt",
          "$rootDir/app/src/main/startup-prof.txt"
      )
    }
}

This is what a baseline profile would look like!

Landroidx/activity/Cancellable;
Landroidx/activity/ComponentActivity;
HSPLandroidx/activity/ComponentActivity;-><init>()V
PLandroidx/activity/ComponentActivity;-><init>()V
HSPLandroidx/activity/ComponentActivity;->access$100(Landroidx/activity/ComponentActivity;)Landroidx/activity/OnBackPressedDispatcher;
PLandroidx/activity/ComponentActivity;->access$100(Landroidx/activity/ComponentActivity;)Landroidx/activity/OnBackPressedDispatcher;
HSPLandroidx/activity/ComponentActivity;->addOnContextAvailableListener(Landroidx/activity/contextaware/OnContextAvailableListener;)V
PLandroidx/activity/ComponentActivity;->addOnContextAvailableListener(Landroidx/activity/contextaware/OnContextAvailableListener;)V
HSPLandroidx/activity/ComponentActivity;->createFullyDrawnExecutor()Landroidx/activity/ComponentActivity$ReportFullyDrawnExecutor;
PLandroidx/activity/ComponentActivity;->createFullyDrawnExecutor()Landroidx/activity/ComponentActivity$ReportFullyDrawnExecutor;
HSPLandroidx/activity/ComponentActivity;->ensureViewModelStore()V
PLandroidx/activity/ComponentActivity;->ensureViewModelStore()V
HSPLandroidx/activity/ComponentActivity;->getActivityResultRegistry()Landroidx/activity/result/ActivityResultRegistry;
PLandroidx/activity/ComponentActivity;->getActivityResultRegistry()Landroidx/activity/result/ActivityResultRegistry;
HSPLandroidx/activity/ComponentActivity;->getDefaultViewModelCreationExtras()Landroidx/lifecycle/viewmodel/CreationExtras;
PLandroidx/activity/ComponentActivity;->getDefaultViewModelCreationExtras()Landroidx/lifecycle/viewmodel/CreationExtras;
HSPLandroidx/activity/ComponentActivity;->getLifecycle()Landroidx/lifecycle/Lifecycle;
PLandroidx/activity/ComponentActivity;->getLifecycle()Landroidx/lifecycle/Lifecycle;
....
....

8. Copy Profile Files:

Copy the generated baseline profile file (e.g., baseline-prof.txt) and the startup profile file (e.g., startup-profile.txt) from the benchmark module's build output (benchmark/build/outputs/managed_device_android_test_additional_output/benchmark/pixel2api34/BaselineProfileGenerator_generateBaselineProfile-baseline-prof.txt) directory to your app module's app/src/main/baseline-prof.txt directory.

How to Test?

Let's switch the build variant to benchmark mode

In your Android project, go to ExampleStartupBenchmark.kt file
Run the tests to measure and analyze your app’s performance metrics.

Since in our application, there was nothing apart from a “hello world” screen, we can compare the median time for timeToInsitialDisplayMs, which can be found that with and without baseline and default compilation, one can notice 1634.2msconverted into 1408.8/1479.9msis ~200msfaster than the previous
By going through hyperlinks, one can find the Android system tracing profiler and hovering over the top-Down chart and not head over to the JIT thread pool. Let's compare CPU Duration for both JIT Thread pool startup profiles with and without a baseline where we can see how time difference in duration by JIT in both cases.
Without Baseline

With Baseline

To verify that the baseline profile is properly attached, build an apk and analyze your apk by checking inside the assets folder. You can confirm whether the baseline file is there or not.

Conclusion:

Developers can optimize app performance by understanding the evolution of Android compilers, implementing benchmarking techniques, and utilizing baseline and startup profiles. Utilizing these practices in the development process empowers developers to create high-performance Android applications that deliver exceptional user experiences across diverse device environments. This makes your app run faster and smoother and provides a better experience.

Sources:

https://www.youtube.com/watch?v=yJm5On5Gp4c
Baseline Profiles → https://developer.android.com/topic/performance/baselineprofiles/overview
Improve app performance with Baseline Profiles → https://goo.gle/baseline-profiles-cod...

About the Author

I’m just a passionate Android developer who loves finding creative elegant solutions to complex problems. Feel free to reach out on LinkedIn to chat about Android development and more.