Handling schema or data type changes in TypeScript

One of my the web apps I’ve been working on recently deals uses a custom eCommerce cart system that users use to make purchase orders with a manufacturing company. Rather than keep a record in a database for every single instance of a cart, we ended up with a pretty elegant solution for saving user carts in localstorage. After planning, designing and wireframing the app, we decided that there wasn’t much benefit to sending all of the cart changes back to the server; this was a strictly business-to-business affair, so we had no need to query empty carts and set up those annoying “you forgot this in your cart” emails. It resulted in a clean and, most importantly, fast cart system that serialized changes to JSON, saved the state to localstorage, and deserialized/restored the cart on the next visit.

The day came, though, as it always does, where we had to make massive changes to the data types representing a cart. But since all of the cart data was stored in our users’ browsers, we had no way to perform data migrations. It’s not so easy when your cart system isn’t using a SQL table — you can’t just send out an UPDATE [Table] SET ... command and move on with your life. Nor can you push an update to migrate old data to new data and assume everyone has been migrated after a week. Who knows when a user is going to log in next? It could be an hour after you publish your changes, it could be tomorrow, it could be a year from now or it could be never. We couldn’t risk a user logging in sometime in the future and the app crashing because it failed to deserialize old data types long-removed from the app.

Rather, we had to restructure the app and the backing types to handle both the new data types and the old data types. There are two methods that I explored for dealing with multiple data types in this application, and both revolve around using TypeScript’s powerful union types:

1. Set up a small migration function around whatever is “feeding” your application data. In our case this was the localstorage mechanism, but it could also be something like a document-style database like CouchDB or MongoDB, or even an API.
2. Refactor any code that deals with your old data type and update it to deal with either the old type or the new type.

Now just looking at these two options, one of them sounds much easier than the other. Would you rather wrap one storage/database/input in a migration function, or would you rather update dozens or hundreds of spots across your codebase to deal with multiple data types?

Sadly, for legacy reasons we weren’t able to immediately use the first option (wrapping the localstorage mechanism in a migration function), and instead we had to update our codebase to deal with multiple data types throughout the app.

I’ll go over both options as they’re relatively simple and employ the same “ideas”. To start with though, we need to establish our data types. For this example, let’s say that we’re dealing with a Trello-style to-do app. Imagine the original data structure looked something like this:

inteface Item {
	name: string;
    /**
     * Optional timestamp indicating when the to-do item was completed.
     */
    completed?: number; 
    /**
     * The name of the Trello-style "board".
     */
    board: string;
}

interface Data {
	items: Item[];
}

So if that’s the original “version 1” of the to-do app’s data types, let’s introduce some changes that will add new a new feature for marking a task as important. We’ll also change the Data type to separate items out into boards rather than having one giant list of unrelated to-do items. I also like to add a version string literal when introducing new type versions, which helps future-proof for even more changes in the future (and ideally your original data types should have a version string literal too, but that’s not always the case.)

Most importantly, though, we need to preserve the original data types and rename them to ItemV1 and DataV1; doing this will ensure we can still work with those original data types.

// Rename Item to ItemV1
interface ItemV1 {
    name: string;
    /**
     * Optional timestamp indicating when the to-do item was completed.
     */
    completed?: number; 
    /**
     * The name of the Trello-style "board".
     */
    board: string;
}

// Rename Data to DataV1
interface DataV1 {
    items: ItemV1[];
}

// Give the new types the original names, so most functions will be updated to use the latest "version"
interface Item {
    name: string;
    /**
     * Optional timestamp indicating when the to-do item was completed.
     */
    completed?: number;
    important: boolean;
    /**
     * A string literal indicating that this Item is version 2 (as opposed to the original version 1).
     */
    version: "v2";
}

interface Data {
    /**
     * In version 2, items are stored according in "boards", which just looks like { "boardName": Item[] }
     */
	boards: {[boardName: string]: Item[]};
    /**
     * A string literal indicating that this Data is version 2 (as opposed to the original version 1).
     */
    version: "v2";
}

A small migration function to map old data types to new data types

With the data types properly versioned, it’s time to look at the first strategy for “migrating” to the latest versions. This is the one that we’d have loved to move to from the start, but again due to legacy reasons that I won’t get into, we couldn’t do it from the start. Regardless, this strategy is most effective when you only have one or two points in your application that actually deal with serializing and deserializing data.

In the application I was working on, the Data types were being serialized to JSON then stored in localStorage, then deserialized and restored on the next page load, using a utility StorageHandler class that looks like this:

class StorageHandler {
    private get storageKey(): string {
        return "_app_data";
    }

    save: (data: Data) => void = data => {
        const serialized = JSON.stringify(data);

        localstorage.setItem(this.storageKey, serialized);
    }

    load: () => Data = () => {
        const serialized = localStorage.getItem(this.storageKey);

        if (serialized === null) {
            // Return an empty Data object
            return {
                items: []
            }
        }

        return JSON.parse(serialized);
    }
}

• Create a migration function that will receive data if either DataV1 | DataV2
• Wrap only deserialization with the migration function. Serialization should only deal with the latest version of data, effectively migrating the user’s data to the latest version once they make a change.

Updating the codebase to deal with both old and new data types

• Rename the new types to DataV2 and ItemV2, then add union types where type Item = ItemV1 | ItemV2 and type Data = DataV1 | DataV2.
• Eventually we were able to set up a small data migration function around the localstorage serialization mechanism, so that any data being deserialized from the browser was marked as DataV1 | DataV2, and the migration function would massage that data into the DataV2 type expected by the rest of the app.
  • We couldn’t do that from the start due to legacy reasons, but once we were able to smoothly switch over to this simple migration function everything became a lot easier.
• Thankfully, TypeScript has made handling old data types so much easier. Rather than changing your old data types, you just version them or rename them, and add your new data type as a separate type.
• Start off by naming the first type “data” or whatever. Then when you need to introduce changes, don’t modify the original type. Rather, rename the original type to DataV1, add a new type called DataV2 for your new… type. Then, add a union type named Data that’s a union of both versions.
  • Now all of your functions that previously dealt with Data have to specifically handle DataV1 and DataV2.
• This obviously works in more situations than just handling deserialization from localstorage. You might have a document database like MongoDB or CouchDB, which is hard (not impossible) to update all of your documents once a change has been made.
• To be clear, I’m not saying this is impossible in JavaScript, but JavaScript doesn’t force you to deal with different types of data.