Android Testing Tutorial: Unit Testing like a True Green Droid
As experienced app developers, as the applications we develop mature, we get a gut feeling that it is the time to start testing. Business rules often imply that the system has to provide stability throughout different releases. We also ideally want to automate the build process and publish the application automatically. For this, we need Adnroid testing tools in place to guarantee that the build is working as expected.
Tests can provide the extra level of confidence about things we build. It is difficult (if not impossible) to build a perfect, bug-free product. Therefore, our goal will be to improve our odds of succeeding on the market by setting up a test suite that will quickly spot newly introduced bugs in our application.
Android testing tutorial

When it comes to Android, and the various mobile platforms in general, app testing can be a challenge. Implementing unit tests and following principles of test-driven development or similar can often feel unintuitive, at the least. Nonetheless, testing is important, and shouldnt be taken for granted or ignored. David, Kent and Martin have discussed the benefits and pitfalls of testing in a series of conversations between themselves compiled in an article titled Is TDD dead?. You can also find the actual video conversations there and get more insight if testing fits your development process and to which extent could you incorporate it, starting now.
In this Android testing tutorial I will walk you through unit and acceptance, regression testing on Android. We will focus on the abstraction of the unit of tests on Android, followed by examples of acceptance testing, with the focus on making the process as fast and simple as possible to shorten developer-QA feedback cycles.

Should I Read It?

This tutorial will explore the different possibilities when it comes to testing Android applications. Developers or project managers who want to better understand the current testing possibilities of the Android platform can decide using this tutorial if they want to take any of the approaches mentioned in this article. However, this is no silver bullet, as the discussion involved in such a topic inherently varies from product to product along with deadlines, codebase quality of code, level of coupling of the system, developers preference in architecture design, projected lifespan of the feature to test, etc.

Thinking in Units: Android Testing

Ideally, we want to test one logical unit/component of an architecture independently. This way we can guarantee that our component works properly for the set of inputs that we expect. The dependencies can be mocked, which will enable us to write tests that execute fast. Furthermore, we will be able to simulate different system states based on the supplied input to the test, covering exotic cases in the process.
The goal of Android unit testing is to isolate each part of the program and show that the individual parts are correct. A unit test provides a strict, written contract that the piece of code must satisfy. As a result, it affords several benefits. Wikipedia

Robolectric

Robolectric is an Android unit testing framework that allows you to run tests inside the JVM on your development workstation. Robolectric rewrites Android SDK classes as theyre being loaded and makes it possible for them to run on a regular JVM, resulting in fast test times. Furthermore, it handles inflation of views, resource loading, and more stuff thats implemented in native C code on Android devices, making the need for emulators and physical devices to run automated tests obsolete.

Mockito

Mockito is a mocking framework that enables us to write clean tests in java. It simplifies the process of creating test doubles (mocks), which are used to replace the original dependencies of a component/module used in production. A StackOverflow answer discusses about the differences between mocks and stubs in fairly simple terms that you can read to learn more.
// you can mock concrete classes, not only interfaces
LinkedList mockedList = mock(LinkedList.class);

// stubbing appears before the actual execution
when(mockedList.get(0)).thenReturn("first");

// the following prints "first"
System.out.println(mockedList.get(0));

// the following prints "null" because get(999) was not stubbed
System.out.println(mockedList.get(999));
Additionally, with Mockito we can verify if a method has been called:
// mock creation
List mockedList = mock(List.class);

// using mock object - it does not throw any "unexpected interaction" exception
mockedList.add("one");
mockedList.clear();

// selective, explicit, highly readable verification
verify(mockedList).add("one");
verify(mockedList).clear();

Now, we know that we can specify action-reaction pairs that define what happens once we execute a specific action on the mocked object/component. Therefore, we can mock entire modules of our application, and for each test case make the mocked module react in a different way. The different ways will reflect the possible states of the tested component and mocked component pair.

Unit Testing

In this section, we will assume MVP (Model View Presenter) architecture. The activities and fragments are the views, models being the repository layer for calls to the database or remote services, and presenter being the brain that binds all these together implementing specific logic to control views, models, and the flow of data through the application.

Abstracting Components

Mocking views and Models

In this Android testing example, we will mock views, models, and repository components, and we will unit test the presenter. This is one of the smallest tests, targeting a single component in the architecture. Furthermore, we will use method stubbing for setting up a proper, testable chain of reactions:
@RunWith(RobolectricTestRunner.class)
@Config(manifest = "app/src/main/AndroidManifest.xml", emulateSdk = 18)
publicclassFitnessListPresenterTest {

private Calendar cal = Calendar.getInstance();

@Mock
private IFitnessListModel model;

@Mock
private IFitnessListView view;

private IFitnessListPresenter presenter;

@Before
publicvoidsetup() {
MockitoAnnotations.initMocks(this);

final FitnessEntry entryMock = mock(FitnessEntry.class);

presenter = new FitnessListPresenter(view, model);
/*
Define the desired behaviour.

Queuing the action in "doAnswer" for "when" is executed.
Clear and synchronous way of setting reactions for actions (stubbing).
*/

doAnswer((new Answer () {
@Override
public Object answer(InvocationOnMock invocation) throws Throwable {
ArrayList items = new ArrayList<>();
items.add(entryMock);

((IFitnessListPresenterCallback) presenter).onFetchAllSuccess(items);
returnnull;
}
})).when(model).fetchAllItems((IFitnessListPresenterCallback) presenter);
}

/**
Verify if model.fetchItems was called once.
Verify if view.onFetchSuccess is called once with the specified list of type FitnessEntry

The concrete implementation of ((IFitnessListPresenterCallback) presenter).onFetchAllSuccess(items);
calls the view.onFetchSuccess(...) method. This is why we verify that view.onFetchSuccess is called once.
*/

@Test
publicvoidtestFetchAll() {
presenter.fetchAllItems(false);
// verify can be called only on mock objects
verify(model, times(1)).fetchAllItems((IFitnessListPresenterCallback) presenter);
verify(view, times(1)).onFetchSuccess(new ArrayList<>(anyListOf(FitnessEntry.class)));
}
}

Mocking the Global Networking Layer with MockWebServer

It is often convenient to be able to mock the global networking layer. MockWebServer allows us to queue responses for specific requests that we execute in our tests. This gives us the chance to simulate obscure responses that we expect from the server, but are not straightforward to reproduce. It allows us to ensure full coverage while writing little additional code.
MockWebServers code repository provides a neat example that you can refer to for a better understanding of this library.

Custom Test Doubles

You can write your own model or respoistory component and inject it into the test by providing a different module to the object graph using Dagger (http://square.github.io/dagger/). We have the option to check if the view state was updated properly based on the data supplied by the mocked model component:
/**
Custom mock model class
*/

publicclassFitnessListErrorTestModelextendsFitnessListModel {

// ...

@Override
publicvoidfetchAllItems(IFitnessListPresenterCallback callback) {
callback.onError();
}

@Override
publicvoidfetchItemsInRange(final IFitnessListPresenterCallback callback, DateFilter filter) {
callback.onError();
}

}
@RunWith(RobolectricTestRunner.class)
@Config(manifest = "app/src/main/AndroidManifest.xml", emulateSdk = 18)
publicclassFitnessListPresenterDaggerTest {

private FitnessActivity activity;
private FitnessListFragment fitnessListFragment;

@Before
publicvoidsetup() {
/*
setupActivity runs the Activity lifecycle methods on the specified class
*/

activity = Robolectric.setupActivity(FitnessActivity.class);
fitnessListFragment = activity.getFitnessListFragment();

/*
Create the objectGraph with the TestModule
*/

ObjectGraph localGraph = ObjectGraph.create(TestModule.newInstance(fitnessListFragment));
/*
Injection
*/

localGraph.inject(fitnessListFragment);
localGraph.inject(fitnessListFragment.getPresenter());
}

@Test
publicvoidtestInteractorError() {
fitnessListFragment.getPresenter().fetchAllItems(false);

/*
suppose that our view shows a Toast message with the specified text below when an error is reported, so we check for it.
*/

assertEquals(ShadowToast.getTextOfLatestToast(), "Something went wrong!");
}

@Module(
injects = {
FitnessListFragment.class,
FitnessListPresenter.class
}, overrides = true,
library = true
)
static class TestModule {
private IFitnessListView view;

privateTestModule(IFitnessListView view){
this.view = view;
}

publicstatic TestModule newInstance(IFitnessListView view){
returnnew TestModule(view);
}

@Provides
public IFitnessListInteractor provideFitnessListInteractor(){
returnnew FitnessListErrorTestModel();
}

@Providespublic IFitnessListPresenter provideFitnessPresenter(){
returnnew FitnessListPresenter(view);
}
}

}

Running Tests

Android Studio

You can easily right click a test class, method, or whole test package and run the tests from the options dialog in the IDE.

Terminal

Running Android app tests from the terminal creates reports for the tested classes in the build folder of the target module. Even more, if you plan to setup an automated build process, you will use the terminal approach. With Gradle, you can run all debug flavored tests by executing the following:
gradle testDebug

Accessing Source Set test from Android Studio Version

Version 1.1 of Android Studio and the Android Gradle plugin brings support for unit testing your code. You can learn more by reading their excellent documentation on it. The feature is experimental, but also a great inclusion since you can now easily switch between your unit tests and instrumentation test source sets from the IDE. It behaves in the same way as if you would switch flavours in the IDE.
Android Unit testing

Easing the Process

Writing Android app tests may not be as fun as developing the original application. Hence, some tips on how to ease the process of writing tests and avoiding common problems while setting up the project will help a long way.

AssertJ Android

AssertJ Android, as you may have guessed from the name, is a set of helper functions that is built with Android in mind. It is an extension to the popular library AssertJ. Functionality provided by AssertJ Android ranges from simple assertions, such as assertThat(view).isGone(), to things as complex as:
assertThat(layout).isVisible()
.isVertical()
.hasChildCount(4)
.hasShowDividers(SHOW_DIVIDERS_MIDDLE)
With AssertJ Android and its extensibility, you are guaranteed a simple, good starting point for writing tests for Android applications.

Robolectric and Manifest Path

While using Robolectric, you may notice that you have to specify the manifest location, and that the SDK version is set to 18. You can do this by including a Config annotation.
@Config(manifest = "app/src/main/AndroidManifest.xml", emulateSdk = 18)
Running tests that require Robolectric from the terminal can introduce new challenges. For example, you may see exceptions like Theme not set. If the tests are executing properly from the IDE, but not from the terminal, you may be trying to run it from a path in the terminal where the specified manifest path can not be resolved. The hard-coded config value for the manifest path may not be pointing to the right location from the point of execution of the command. This can be solved through the use of custom runners:
publicclassRobolectricGradleTestRunnerextendsRobolectricTestRunner {
publicRobolectricGradleTestRunner(Class testClass) throws InitializationError {
super(testClass);
}

@Override
protected AndroidManifest getAppManifest(Config config) {
String appRoot = "../app/src/main/";
String manifestPath = appRoot + "AndroidManifest.xml";
String resDir = appRoot + "res";
String assetsDir = appRoot + "assets";
AndroidManifest manifest = createAppManifest(Fs.fileFromPath(manifestPath),
Fs.fileFromPath(resDir),
Fs.fileFromPath(assetsDir));
return manifest;
}
}

Gradle Configuration

You can use the following to configure Gradle for unit testing. You may need to modify the dependency names and versions required based on your project needs.
// Robolectric
testCompile 'junit:junit:4.12'
testCompile 'org.mockito:mockito-core:1.9.5'
testCompile 'com.squareup.dagger:dagger:1.2.2'
testProvided 'com.squareup.dagger:dagger-compiler:1.2.2'

testCompile 'com.android.support:support-v4:21.0.+'
testCompile 'com.android.support:appcompat-v7:21.0.3'

testCompile('org.robolectric:robolectric:2.4') {
exclude module: 'classworlds'
exclude module: 'commons-logging'
exclude module: 'httpclient'
exclude module: 'maven-artifact'
exclude module: 'maven-artifact-manager'
exclude module: 'maven-error-diagnostics'
exclude module: 'maven-model'
exclude module: 'maven-project'
exclude module: 'maven-settings'
exclude module: 'plexus-container-default'
exclude module: 'plexus-interpolation'
exclude module: 'plexus-utils'
exclude module: 'wagon-file'
exclude module: 'wagon-http-lightweight'
exclude module: 'wagon-provider-api'
}

Robolectric and Play Services

If you are using Google Play Services, you will have to create your own integer constant for the Play Services version in order for Robolectric to work properly in this application configuration.
<meta-data
android:name="com.google.android.gms.version"
android:value="@integer
/gms_version"
tools:replace="android:value" />

Robolectric Dependencies to Support Libraries

Another interesting testing problem is that Robolectric isnt able to reference support libraries properly. The solution is to add a project.properties file to the module where the tests are. For example, for the Support-v4 and AppCompat libraries the file should contain:
android.library.reference.1=../../build/intermediates/exploded-aar/com.android.support/support-v4/21.0.3
android.library.reference.2=../../build/intermediates/exploded-aar/com.android.support/appcompat-v7/21.0.3

Acceptance/Regression Testing

Acceptance/Regression testing automates part of the final step of testing on a real, 100% percent Android environment. We do not use mocked Android OS classes at this level - the tests are run on real devices and emulators.
android acceptance and regression testing

These circumstances make the process much more unstable due to the variety of physical devices, emulator configurations, device states, and feature sets of each device. Furthermore, it is highly dependent on the version of the operating system and the screen size of the phone to decide how the content will be displayed.
It is a bit complex to create the right test that passes on a wide range of devices, but as always you should dream big yet start small. Creation of tests with Robotium is an iterative process. With a few tricks, it can be simplified a lot.

Robotium

Robotium is an open source Android test automation framework that has been in existence since January 2010. It is worth mentioning that Robotium is a paid solution, but comes with a fair free trial.
To speed up the process of writing Robotium tests, we will move away from manual test writing to test recording. The trade-off is between quality of code and speed. If you are making heavy changes to your user interface, you will benefit a lot from the test recording approach and being able to record new tests quickly.
Testdroid Recorder is a free test recorder that creates Robotium tests as it records the clicks you perform on the user interface. Installing the tool is super easy, as described in their documentations accompanied with a step-by-step video.
Since Testdroid Recorder is an Eclipse plugin and we are referring to Android Studio throughout this article, it would ideally be a reason of concern. However, in this case it is not a problem, as you can use the plugin directly with an APK and record the tests against it.
Once you create the tests, you can copy and paste them in Android Studio, together with any dependency that Testdroid recorder requires, and you are set to go. The recorded test would look something like the class below:
publicclassLoginTestextendsActivityInstrumentationTestCase2<Activity> {

privatestaticfinal String LAUNCHER_ACTIVITY_CLASSNAME = "com.toptal.fitnesstracker.view.activity.SplashActivity";
privatestatic Class launchActivityClass;
static {
try {
launchActivityClass = Class.forName(LAUNCHER_ACTIVITY_CLASSNAME);
} catch (ClassNotFoundException e) {
thrownew RuntimeException(e);
}
}
private ExtSolo solo;

@SuppressWarnings("unchecked")
publicLoginTest() {
super((Class ) launchActivityClass);
}

// executed before every test method
@Override
publicvoidsetUp() throws Exception {
super.setUp();
solo = new ExtSolo(getInstrumentation(), getActivity(), this.getClass()
.getCanonicalName(), getName());
}

// executed after every test method
@Override
publicvoidtearDown() throws Exception {
solo.finishOpenedActivities();
solo.tearDown();
super.tearDown();
}

publicvoidtestRecorded() throws Exception {
try {
assertTrue(
"Wait for edit text (id: com.toptal.fitnesstracker.R.id.login_username_input) failed.",
solo.waitForEditTextById(
"com.toptal.fitnesstracker.R.id.login_username_input",
20000));
solo.enterText(
(EditText) solo
.findViewById("com.toptal.fitnesstracker.R.id.login_username_input"),
"user1@gmail.com");
solo.sendKey(ExtSolo.ENTER);
solo.sleep(500);
assertTrue(
"Wait for edit text (id: com.toptal.fitnesstracker.R.id.login_password_input) failed.",
solo.waitForEditTextById(
"com.toptal.fitnesstracker.R.id.login_password_input",
20000));
solo.enterText(
(EditText) solo
.findViewById("com.toptal.fitnesstracker.R.id.login_password_input"),
"123456");
solo.sendKey(ExtSolo.ENTER);
solo.sleep(500);
assertTrue(
"Wait for button (id: com.toptal.fitnesstracker.R.id.parse_login_button) failed.",
solo.waitForButtonById(
"com.toptal.fitnesstracker.R.id.parse_login_button",
20000));
solo.clickOnButton((Button) solo
.findViewById("com.toptal.fitnesstracker.R.id.parse_login_button"));
assertTrue("Wait for text fitness list activity.",
solo.waitForActivity(FitnessActivity.class));
assertTrue("Wait for text KM.",
solo.waitForText("KM", 20000));

/*
Custom class that enables proper clicking of ActionBar action items
*/

TestUtils.customClickOnView(solo, R.id.action_logout);

solo.waitForDialogToOpen();
solo.waitForText("OK");
solo.clickOnText("OK");

assertTrue("waiting for ParseLoginActivity after logout", solo.waitForActivity(ParseLoginActivity.class));
assertTrue(
"Wait for button (id: com.toptal.fitnesstracker.R.id.parse_login_button) failed.",
solo.waitForButtonById(
"com.toptal.fitnesstracker.R.id.parse_login_button",
20000));
} catch (AssertionFailedError e) {
solo.fail(
"com.example.android.apis.test.Test.testRecorded_scr_fail",
e);
throw e;
} catch (Exception e) {
solo.fail(
"com.example.android.apis.test.Test.testRecorded_scr_fail",
e);
throw e;
}
}
}
If you look closely, you will notice how much of the code is rather straight-forward.
When recording tests, dont go scarce on wait statements. Wait for dialogs to appear, activities to appear, texts to appear. This will guarantee that the activity and the view hierarchy are ready to be interacted with when you perform the action on the current screen. At the same time, take screenshots. Automated tests are usually unattended, and screenshots are one of the ways you get to see what actually happened during those tests.
Whether tests pass or fail, reports are your best friend. You can find them under the build directory module/build/outputs/reports:
test reporting

In theory, the QA team could record tests and optimize them. By putting effort in a standardized model for optimizing test cases, it could be done. When you normally record tests, you always have to tweak a couple of things to make it work flawlessly.
Finally, to run these tests from Android Studio, you can select them and run as you would run unit tests. From the terminal, it is a one-liner:
gradle connectedAndroidTest

Performance of Testing

Android unit testing with Robolectric is extremely fast, because it runs directly within the JVM on your machine. Compared to that, acceptance testing on emulators and physical devices is a lot slower. Depending on the size of flows that you are testing, it can take anywhere from a few seconds to a few minutes per test case. The acceptance test phase should be used as part of an automated build process on a continuous integration server.
Speed can be improved by parallelization on multiple devices. Check out this great tool from Jake Wharton and the guys at Square http://square.github.io/spoon/. It has some nice reporting, too.

The Takeaway

There are a variety of Android testing tools available, and as the ecosystem matures, the process of setting up a testable environment and writing tests will become easier. There are still more challenges to tackle, and with a wide community of developers working on daily problems, there is a lot of room for constructive discussions and fast feedback.
Use the approaches described in this Android testing tutorial to guide you in tackling the challenges ahead of you. If and when you run into problems, check back with this article or the references linked within for solutions to known problems.
In a future post, we will discuss parallelization, build automation, continuous integration, Github/BitBucket hooks, artifact versioning, and the best practices for managing massive mobile application projects in greater depth.



Posted on :

About the Author

Demir Selmanovic

Demir is a developer and project manager with over 15 years of professional experience in a wide range of software development roles. He excels as a solo developer, team member, team leader, or manager of multiple distributed teams. He works closely with clients to define ideas and deliver products.

« Previous
The Vital Guide to Android Interviewing


x