Generic persistent data storage in Android using Jetpack DataStore

Introduction

One way or the other, any modern Android application stores some user or config data locally on the device. In the past, developers relied on the SharedPreferences API to store simple data in key-value pairs.

Where SharedPreferences API fails to shine is in its synchronous API for read and write operations. Since Android frowns at performing non-UI work on the main thread, this is not safe to use.

In this article, you will learn how to use the DataStore API with generic persistent storage. This approach will let us create a storage class where we can specify any data type we wish to save as a key-value pair to the device.

We’ll cover the following topics:

• Advantages of using Jetpack DataStore
• Setting up a sample Android application
• Creating a Kotlin storage interface
• Creating a concrete implementation of the storage interface
• Implementing the getAll operation
• Implementing the insert operation
• Implementing the get operation
• Implementing the clearAll operation
• Creating the model class and in-memory data source
• How to inject dependencies with Koin
• Initializing Koin to prepare dependencies
• Benefits of generic persistent storage with Android DataStore

Advantages of using Jetpack DataStore

• DataStore is fully asynchronous, using Kotlin coroutines
• Read and write operations are done in the background, without fear of blocking the UI
• With coroutines, there are mechanisms in place for error signaling when using DataStore

Setting up a sample Android application

In this demo, we will create a sample application to fetch the application’s configurations from an in-memory source and save them on the device using DataStore.

There are a few prerequisites before we can get started:

• Basic knowledge of Android mobile development and Kotlin
• Android Studio installed on your PC

Let’s start by creating an empty Android Studio project.

Copy and paste the following dependencies into your app-level build.gradle file.

implementation "androidx.datastore:datastore-preferences:1.0.0"
implementation "io.insert-koin:koin-android:3.1.4"
implementation 'com.google.code.gson:gson:2.8.7'

Alongside the dependency for DataStore are the extra koin and gson dependencies, which are for dependency injection and serialization/deserialization, respectively.

After inserting these dependencies, Android Studio will prompt you to sync the project. This typically takes a few seconds.

Creating a Kotlin storage interface

Create a Kotlin interface file, like so.

interface Storage<T> {

    fun insert(data: T): Flow<Int>

    fun insert(data: List<T>): Flow<Int>

    fun get(where: (T) -> Boolean): Flow<T>

    fun getAll(): Flow<List<T>>

    fun clearAll(): Flow<Int
}

We use a storage interface to define the actions for the persistent data storage. In other words, it is a contract the persistent storage will fulfill. Any data type we intend to associate with the interface should be able to carry out all four operations in the interface we created.

Creating a concrete implementation of the storage interface

PersistentStorage is the concrete implementation of Storage interface we defined in the previous step.

class PersistentStorage<T> constructor(
    private val gson: Gson,
    private val type: Type,
    private val dataStore: DataStore<Preferences>,
    private val preferenceKey: Preferences.Key<String>
) : Storage<T>

You will observe by now that we are taking advantage of generics in Storage and PersistentStorage. This is done to achieve type safety. If your code is relying on generic persistent storage to store data, only one data type will be associated with a particular instance of Storage.

There are also a number of object dependencies required:

1. gson: As previously mentioned, this will be used for serialization/deserialization
2. type: Our implementation gives the user the flexibility to save more than one piece of data of the same type — and with great power comes great responsibility. Writing and reading a list with GSON will result in corrupted or lost data because Java doesn’t yet provide a way to represent generic types, and GSON cannot recognize which type to use for its conversion at runtime, so we use a type token to effectively convert our objects to a JSON string and vice versa without any complications
3. Preference Key: This is an Android Jetpack DataStore-specific object; it is basically a key for saving and retrieving data from DataStore
4. DataStore: This will provide APIs for writing to and reading from the preferences

Implementing the getAll operation

...
fun getAll(): Flow<List> {
    return dataStore.data.map { preferences ->
        val jsonString = preferences[preferenceKey] ?: EMPTY_JSON_STRING
        val elements = gson.fromJson<List>(jsonString, typeToken)
        elements
    }
}
...

DataStore.data returns a flow of preferences with Flow<Preferences>, which can be transformed into a Flow<List<T>> using the map operator. Inside of the map block, we first attempt to retrieve the JSON string with the preference key.

In the event that the value is null, we assign EMPTY_JSON_STRING to jsonString. EMPTY_JSON_STRING is actually a constant variable, defined like so:

private const val EMPTY_JSON_STRING = "[]"

GSON will conveniently recognize this as a valid JSON string, which represents an empty list of the specified type. This approach is more logical, rather than throwing some exception that could potentially cause a crash in the app. I am sure we don’t want that happening in our apps 🙂

Implementing the insert operation

fun insert(data: List<T>): Flow<Int> {
    return flow {
        val cachedDataClone = getAll().first().toMutableList()
        cachedDataClone.addAll(data)
        dataStore.edit {
            val jsonString = gson.toJson(cachedDataClone, type)
            it[preferenceKey] = jsonString
            emit(OPERATION_SUCCESS)
        }
    }
}

To write data to DataStore, we call edit on Datastore. Within the transform block, we edit the MutablePreferences, as shown in the code block above.

To avoid overwriting the old data with the new, we create a list that contains both old data and new data before modifying MutablePreferences with the newly created list.

n.b., I opted to use method overloading to insert a single or a list of data over a vararg parameter because varargs in Kotlin require extra memory when copying the list of data to an array.

Implementing the get operation

fun get(where: (T) -> Boolean): Flow {
    return getAll().map { cachedData ->
        cachedData.first(where)
    }
}

In this operation, we want to get a single piece of data from the store that matches the predicate where. This predicate is to be implemented on the client side.

Implementing the clearAll operation

fun clearAll(): Flow<Int> {
    return flow {
        dataStore.edit {
            it.remove(preferenceKey)
            emit(OPERATION_SUCCESS)
        }
    }
}

As the name implies, we want to wipe the data that is associated with the preference key. emit(OPERATION_SUCCESS) is our way of notifying the client of a successful operation.

At this point, we have done justice to the generic storage APIs. Up next, we’ll set up the model class and an in-memory data source.

Creating the model class and in-memory data source

Create a Config data class, like so:

data class Config(val type: String, val enable: Boolean)

To keep things simple, this data class only captures a config type and its corresponding toggle value. Depending on your use case, your config class can describe many more actions.

class DataSource {

    private val _configs = listOf(
        Config("in_app_billing", true),
        Config("log_in_required", false),
    )

    fun getConfigs(): Flow<List<Config>> {
        return flow {
            delay(500) // mock network delay
            emit(_configs)
        }
    }
}

For lack of an actual server to connect to, we have our configs stored in-memory and retrieved when needed. We have also included a delay to mock an actual network call.

How to inject dependencies with Koin

While this article is focused on creating a minimalistic demo Android app, it is okay to adopt some modern practices. We will implement the code for fetching configs via a ViewModel and provide dependencies to objects where necessary using koin.

What is Koin?

Koin is a powerful Kotlin dependency injection framework. It has simple APIs and is relatively easy to set up.

Create a ViewModel class

class MainViewModel(
    private val dataSource: DataSource,
    private val configStorage: Storage<Config>
) : ViewModel() {

    init {
        loadConfigs()
    }

    private fun loadConfigs() = viewModelScope.launch {
        dataSource
            .getConfigs()
            .flatMapConcat(configStorage::insert)
            .collect()
    }
}

Here, we fetch the configs from a data source and save them to our DataStore preferences.
The intention is to be able to retrieve those configs locally without having to make additional network calls to the server. The most obvious place to initiate this request would be at app launch.

Define your koin modules like so:

private val Context.dataStore: DataStore<Preferences> by preferencesDataStore(name = "com.enyason.androiddatastoreexample.shared.preferences")

val dataAccessModule = module {

    single<Storage<Config>> {
        PersistentStorage(
            gson = get(),
            type = object : TypeToken<List<Config>>() {}.type,
            preferenceKey = stringPreferencesKey("config"),
            dataStore = androidContext().dataStore
        )
    }

    single { Gson() }

    viewModel {
        MainViewModel(
            dataSource = DataSource(),
            configStorage = get()
        )
    }

}

We have now delegated the heavy lifting to Koin. We no longer have to worry how the objects are being created — Koin handles all of that for us.

The single definition tells Koin to create only one instance of the specified type throughout the lifecycle of the application. The viewModel definition tells Koin to create only an object type that extends the Android ViewModel class.

Initializing Koin to prepare dependencies

We need to initialize Koin to prepare our dependencies before our app requests them. Create an Application class, like so:

class App : Application() {

    override fun onCreate() {
        super.onCreate()
        startKoin {
            androidContext(this@App)
            modules(dataAccessModule)
        }
    }
}

We have finally wired up all the pieces together, and our project should now work as expected. Kindly check out this GitHub repo for the complete project setup.

Benefits of generic persistent storage with Android DataStore

• DataStore APIs are powered by Kotlin coroutines under the hood, which makes the generic persistent storage thread safe, unlike the SharedPreferences API
• Read and write logic are written only once for any object type
• Assurance of type safety: Storage<Config> is sure to retrieve only the data of Config type
• PreferenceHelper classes, which are intended to manage app preferences, usually result in monolith classes, which is a bad software engineering practice. With the generic approach discussed in this article, you can achieve more with less code
• We can effectively unit test PersistentStorage<T>

Conclusion

Implementing generic persistent storage is an elegant way of managing data with Android DataStore. The gains as I have discussed above outweigh the traditional approach on Android with SharedPreference. I hope you liked this tutorial 😊