No way to test "System needs resources" situation using ActivityScenario

[seadowg]

Description

I'll refer to the lingo used in the recent Google I/O video Best practices for saving UI state on Android as that coincidently covered the situation I'm describing here. It lays out three ways of "losing app state":

1. Configuration changes
2. System needs resources
3. App dismissal

Using ActivityScenario it's currently possible to write tests for 1 and 3, but not for 2.

Here's an example of testing 1:

val activityScenario = ActivityScenario.launch(Activity::class.java)

// Do something

activityScenario.recreate() // Simulates a config change

// Check something

...and here's an example of testing 3:

val activityScenario1 = ActivityScenario.launch(Activity::class.java)

// Do something

val activityScenario2 = ActivityScenario.launch(Activity::class.java) // Simulates a dismissal and a reopen

// Check something

As far as I understand, when 2 occurs, the following things happen:

1. onSaveInstanceState is called on the Activity
2. The Activity is destroyed
3. The process is killed

When the user navigates back after 2 the Activity is created again with the Bundle earlier created by onSavedInstanceState. The Activity and its Fragments (or potentially a whole back stack) is still being "recreated" his is distinct from 1 in that the process (and any state it owns) is being destroyed (meaning loss of Application level, ViewModel or any unfortunately static state). As an aside, a good way to reproduce this situation locally (without changing background limits or using a low memory phone) is to remove a permission from an app in system settings.

We actually can test this scenario using Robolectric in a local test:

val activityController1 = Robolectric.buildActivity(Activity::class.java)
        activityController1.setup()

// Do something

val outState = Bundle()
activityController1
    .pause()
    .saveInstanceState(outState)
    .stop()
    .destroy()
    .close()

val activityController2 = Robolectric.buildActivity(Activity::class.java)
activityController2.setup(outState)

// Check something

Given this, it seems like it'd be possible to add support for this situation to LocalActivityInvoker and build an API around it for ActivityScenario, but I'm not sure how plausible it is for InstrumentationActivityInvoker which would be good to discuss here. It could be that this isn't something that AndroidX Test will ever support and will forever be left up to "local" tests, but if so that would be good to call out so that Robolectric support can be documented and/or smoothed out.

AndroidX Test and Android OS Versions

AndroidX Test 1.5.0, any version of Android

[seadowg]

I've expanded on my examples in a blog post here.

[seadowg]

Bumping this as I feel like it's a pretty important omission when the new Android testing docs suggest the following about Robolectric:

In general, if you implement a testable architecture you shouldn't need to use Robolectric for unit testing as your code should be testable in isolation, with no dependencies on the Android framework.