Common Automation Testing Frameworks for Android

1. Monkey is a testing tool that comes with the Android SDK. During testing, it sends pseudo-random user event streams to the system, such as key inputs, touchscreen inputs, gestures, etc., to perform stress testing on the application being developed, and it also outputs logs. In fact, this tool can only perform some stress tests on the program. Due to the randomness of the test events and data, it cannot be customized, so it has significant limitations.

2. MonkeyRunner is also a testing tool provided by the Android SDK. Strictly speaking, MonkeyRunner is actually an API toolkit that is more powerful than Monkey, allowing for the writing of test scripts to customize data and events. The downside is that the scripts are written in Python, which requires a higher skill level from testers, leading to a significant learning curve.

3. Instrumentation is an early Android automation testing tool provided by Google. Although JUnit could also test Android at that time, Instrumentation allows for more complex tests on applications, even at the framework level. With Instrumentation, you can simulate key presses, releases, screen clicks, scrolls, and other events. Instrumentation achieves these functions by running the main program and the test program in the same process. You can think of Instrumentation as a component similar to an Activity or Service without a UI, monitoring your main program during its execution. The downside is that it requires testers to have a higher coding ability, a certain understanding of Android-related knowledge, and the need to configure the AndroidManifest.xml file, and it cannot span multiple apps.

4. UiAutomator is also an automation testing framework provided by Android that basically supports all Android event operations. Compared to Instrumentation, it does not require testers to understand the implementation details of the code (you can use UiAutomatorviewer to capture the properties of controls on the app page without looking at the source code). Based on Java, the structure of the test code is simple, easy to write, and has a low learning cost. Once compiled, the tests can run on all devices or emulators and can span apps (for example: many apps have options to select a photo album or open the camera to take a picture, which is cross-app testing). The downside is that it only supports SDK 16 (Android 4.1) and above and does not support Hybrid Apps or Web Apps.

5. Espresso is Google’s open-source automation testing framework. Compared to Robotium and UIAutomator, its features are smaller in scale, more concise, and the API is more precise, making it simple to write test code and easy to get started quickly. Since it is based on Instrumentation, it cannot span apps. It can be used in conjunction with Android Studio to write simple test examples.

6. Selendroid is also a testing framework based on Instrumentation that can test Native Apps, Hybrid Apps, and Web Apps, but there is less information available online and the community is not very active.

7. Robotium is also a testing framework based on Instrumentation that is currently widely used both domestically and internationally, with a lot of available resources and an active community. The downside is that testers need to have a certain foundation in Java and understand the basic components of Android, and it cannot span apps.

8. Athrun is a mobile testing framework/platform developed by Taobao that supports both iOS and Android. The Android part is also based on Instrumentation, extending the original ActivityInstrumentationTestCase2 class in Android, providing a complete set of object-oriented APIs. Here is a detailed introduction.

9. Appium is a relatively popular framework recently, and the community is very active. This framework is likely the most powerful.

• Its advantages:

• Open source;

• Supports Native Apps, Hybrid Apps, and Web Apps;

• Supports Android, iOS, and Firefox OS;

• The server is also cross-platform, so you can use Mac OS X, Windows, or Linux;

• Its philosophy is:

• Using Appium for automated testing does not require recompiling the app;

• Supports many languages for writing test scripts, including Java, JavaScript, PHP, Python, C#, Ruby, and other mainstream languages;

• There is no need to reinvent the wheel for automated testing because it extends WebDriver. (WebDriver is a simple and fast automated testing framework for testing WebApps, so testers with web automation experience can get started directly);

• Mobile automated testing should be open source;

• Its design philosophy:

• Client/Server architecture, where the server listens for commands sent from the client and translates these commands to send to the mobile device or emulator, which then responds. Because of this architecture, the client can use Appium client libraries to write test scripts in various languages, and the server can be deployed on a server, even a cloud server.

• Session: Each client connected to the server will have a session ID, and the client must use this session ID to send commands to the server, as this session ID represents the browser or mobile device emulator you have opened. You can even open multiple sessions to test different devices or emulators simultaneously.

• Desired Capabilities: This is essentially a key-value pair that sets some testing-related information to inform the server whether we need to test iOS, Android, or WebApp, etc.

• The Appium server is written in Node.js, so it can be installed directly using NPM.

• Appium clients provide a GUI on Mac OS and Windows, so there is no need to install Node.js, making it convenient for testers to operate.

• Related limitations:

• If you are using Appium on Windows, you cannot use precompiled .app files specifically for OS X, because Appium relies on OS X-specific libraries to support iOS testing, so you cannot test iOS apps on the Windows platform. This means you can only run iOS tests on a Mac.

• Summary:

• On the iOS side, it encapsulates UIAutomation; for Android 4.2 and above, it uses UiAutomator, and for Android 2.3 to 4.1, it uses Instrumentation. This means that Appium encapsulates both UiAutomator and Instrumentation. Therefore, Appium has all the advantages of the above frameworks: cross-app support, Native App, Hybrid App, Web App support, and it supports multiple languages for writing your test scripts.