[Proposal] Fallback Mechanisms

[matt-ramotar]

TLDR

Introduce Superstore and Warehouse to handle data retrieval and fallback mechanisms when a primary data source fails or returns an error.

Abstract

When a primary data source fails or returns an error, it is desirable to have a fallback mechanism in place. One proposed solution is to introduce two new classes, Warehouse and Superstore, to handle the fallback mechanism. The Warehouse contains the fallback data source and provides a simple API for fetching the data. The Superstore coordinates the Store and Warehouse, attempting to fetch data from the Store first, and if that fails, it will fetch data from the Warehouse.

Fallback Scenario	Description	Secondary Data Source
Network failure/server downtime	When the primary data source (e.g., remote server or API) is temporarily unavailable, the application can switch to a local cache or hardcoded data to continue providing data to the users.	Local cache, local databases, in-memory storage, file-based storage
API rate limiting	When an application exceeds the rate limits of an API, it may need to switch to another data source (such as a local cache or a secondary API) until the rate limits reset.	Local cache, secondary API
Version fallback	In some cases, an application may need to fall back to an older version of the data or API if the current version is not supported, contains bugs, or is deprecated.	Local cache, local databases
Data validation failure	If the data received from the primary source fails validation or is incomplete, the application may use an alternative data source to ensure it provides accurate and complete information to users.	Local cache, local databases, file-based storage
Geo-redundancy	In case of regional network issues or data center outages, applications may automatically switch to a different data center or region to maintain availability.	Secondary data center, CDN
Scheduled maintenance	During planned maintenance of a primary data source, an application can switch to a backup or alternative source to minimize downtime and ensure continuous service.	Backup data source, file-based storage
Data synchronization issues	When data synchronization between multiple sources is delayed or interrupted, the application may switch to another source with more up-to-date information.	Secondary data source, local databases, in-memory storage

Problem

For some use cases, it is desirable to have a fallback mechanism in place for when the Store encounters an error or is unable to fetch data. Currently, there is no built-in mechanism to handle such scenarios, leaving developers to implement custom solutions.

Alternatives considered

1. Modify the existing Store class: We could modify the Store class to include an additional fallback mechanism. However, this approach would increase the complexity of the Store class and could break existing implementations.

2. Create a new FallbackStore class: We could create a new FallbackStore class that inherits from the Store class and provides a fallback mechanism. This approach would keep the existing Store class intact but may result in code duplication and maintenance overhead.

3. Provide guidance to users: By providing comprehensive documentation on handling fallback scenarios, developers can implement a fallback mechanism that fits their specific use case while avoiding unnecessary complexity or potential issues with modifying existing code.

Proposed solution

My proposal is to introduce two new classes, Warehouse and Superstore, to handle the fallback mechanism:

Warehouse

The Warehouse will contain hardcoded data or any other fallback data source. It will provide a simple API for fetching the fallback data.

interface Warehouse<Key : Any, Output : Any> {
    suspend fun get(key: Key): Output?
}

class HardcodedWarehouse<Key : Any, Output : Any>(private val data: Map<Key, Output>) : Warehouse<Key, Output> {
    override suspend fun get(key: Key): Output? {
        return data[key]
    }
}

Superstore

A class that coordinates the Store and Warehouse. The Superstore will first attempt to fetch data from the Store. If the Store fails or returns an error, the Superstore will fetch the data from the Warehouse.

class Superstore<Key : Any, Output : Any>(
    private val store: Store<Key, Output>,
    private val warehouses: List<Warehouse<Key, Output>>
) {
    suspend fun get(key: Key): Output {
        val storeResponse = store.stream(StoreReadRequest.fresh(key)).firstOrNull()
        if (storeResponse is StoreReadResponse.Data) {
            return storeResponse.value
        } else {
            for (warehouse in warehouses) {
                val fallbackData = warehouse.get(key)
                if (fallbackData != null) {
                    return fallbackData
                }
            }
        }
    }
}

[matt-ramotar]

@digitalbuddha Wanted to get this out of my head into an issue. My hunch is we will want to provide guidance through documentation and not change Store

[digitalbuddha]

Yup this looks great. I think it opens up new use cases like managing feature flags where on first launch you don't have network and need the local default flag value. I'll read in way more detail but on first glance I love the direction you are going

[yigit]

I think this looks good. I have 1 question on the first use case listed:
Network failure/server downtime
Given that we allow a local persistent storage already , how do we expect developers to coordinate the two? Does the warehouse fallback to that same storage or is it only expected to be used for cases where the local persistent backend of Store really doesn't have the data.

[matt-ramotar]

Thanks for reviewing! There is an outstanding feature request for enabling SOT as a fallback on failure to fetch fresh data (#394). My thinking has been fallbacks would be available regardless of whether the Store has data. It's possible to use the SOT as a warehouse. For example take a look at this test case. So a user could choose to provide the SOT as a warehouse, or for instance a secondary API and then SOT bindings.

[matt-ramotar]

Hmm - But yeah I did not consider:

1. Conditional error handling - For example, if SOT is used as a warehouse, the user probably wants Superstore to swallow error when Store is empty. Currently we always throw if all fallbacks fail to find data.
2. Conditional fall through - For example, if Store is empty, maybe a user wants to only fall back on a secondary API.

Thoughts?

[yigit]

Hmm, it sounds a bit weird for fresh to fallback to SOT since it is all about not using SOT data. Otoh, I guess it would be convenient to provide a flag (maybe we want Store.fresh(fallbackToSourceOfTruth = false) ).

I feel like SOT serving as Warehouse is a bit weird. The data flow graph looks complicated, as in, there are 2 paths where it might fetch from SOT. If we are adding warehouse just for fresh, i think letting fresh accept a fallback parameter is a better solution.

I think there is still the case for the WareHouse when SOT is empty, but in that case, should this be something that goes under SOT instead of above Store? e.g. what if you could create SOT with a fallback Warehouse ? I'm wondering if that is a more accurate way to model it (and then Warehouse should not be using SOT). Or we could even create an API on SOT builder that lets you chain multiple SOTs?

e.g.

           val warehouse = SourceOfTruth.of<String, Page>(...)
           val sourceOfTruth = SourceOfTruth.of<String, Page>(
                nonFlowReader = { key -> pagesDatabase.get(key) },
                writer = { key, page -> pagesDatabase.put(key, page) },
                delete = null,
                deleteAll = null,
                warehouse = warehouse // or maybe call it fallback?
            )

that way, it can even be chained indefinitely :) I don't think this solves some options listed above (e..g. Geo case) but maybe the solution for that is to also allow chaining Fetchers? Both of these would also be just convenience APIs since these can all be done w/ coroutine operators (i think)

[matt-ramotar]

Thanks, very helpful! Chaining fetchers makes a lot of sense to me. WIP - 2d1cb0b. Will finish refactoring and documenting tomorrow

[matt-ramotar]

Closed with #545

[digitalheir]

a bit weird for fresh to fallback to SOT since it is all about not using SOT data

I think the documentation for the ReadRequest functions should be updated to make this abundantly clear. It currently says:

refresh - if true then return fetcher (new) data as well (updating your caches)

To me the portion "as well" signals that the resulting flow emits first the cached value (if any), then the network data (if any) -- and potentially races the two sources? It's unclear as an end user.