How can we update potentially many Read Model Projections based on an Event that is not within the native context?

Question

I have a Shop which sells Items for a given Price.

To be clear, this is in the context of a video game, not an enterprise ERP system. The game operator can create Shops that Players can use to buy supplies, weapons, tools, etc. Players also have the ability to create their own Shops, ideally using the same rails the game operator uses.

Additionally, this question doesn't focus on the backing persistence model. My goal is to provide just enough of an interface that game operators can decide which persistence mechanism they want to use (NoSQL, MySQL, JSON files, Text files, etc).

Let's say we had a Shop that sold three Items; Potions, Buckets, and Shields. Each one has a different Price. Additionally, we'll say that the Potion Item was incorrectly named "Potionn" (something that will be fixed later).

The User Interface would present the following to the user:

Item Name          Price
Potionn            $100
Bucket             $10
Shield             $40

The Read Model Projection to represent this Shop could look something like

{
    "shop_id": "1",
    "items": [
        {
            "item_id": "potion",
            "item_name": "Potionn",// misspelled name
            "price": {
                "currency": "dollars",
                "amount": 100
            }
        },
        {
            "item_id": "bucket",
            "item_name": "Bucket",
            "price": {
                "currency": "dollars",
                "amount": 10
            }
        },
        {
            "item_id": "shield",
            "item_name": "Shield",
            "price": {
                "currency": "dollars",
                "amount": 40
            }
        }
    ]
}

The Event Stream that fed this Projection could look something like

Item Created ("potion")
Item Created ("bucket")
Item Created ("shield")
Item Named ("potion", "Potionn")// misspelled name
Item Named ("bucket", "Bucket")
Item Named ("shield", "Shield")
Shop Created (1)
Item Added to Shop (1, "potion", (100, "dollars"))
Item Added to Shop (1, "bucket", (10, "dollars"))
Item Added to Shop (1, "shield", (40, "dollars"))

The question is in regards to what happens when an Item Name is changed, after its been added to a Shop.

Item Named ("potion", "Potion")

Here, we change the name to correct an issue (the misspelling). The Projection subsequently needs to be changed, to provide the correct name to the user. If there was only one Shop that sold Potions, we would only need to update one Projection. But, if there are hundreds or thousands of Shops that sell Potions, then all of those will need to be updated in response to the corrected Item Named Event.

Some potential solutions I've come up with are:

1. Just iterate through every Shop, and update the name if it has a "potion" Item. Problems include having to know all of the Shops that exist (keeping a separate list of Shop Ids), and checking all Shops, even if they don't include a Potion Item.
2. Maintain a separate index of all Shops that contain a given Item. The base projection used by the User Interface is keyed on Shop Id, and contains a list of Items in that Shop. This separate index would be keyed on Item Id, and contain a list of Shops that contain that Item. This would reduce the amount of searching needed, only going through Shops that actually had the Potion Item, instead of all Shops that exist. The problem though is having to maintain two separate models, one actually used by the user interface, and another one making reacting to certain events easier; in this case, reducing the amount of Shop Projections needing to be found, and updated.
3. Maintain an internal list of Item Names, and Join this list with a slimmed down Projection of Shop Items when queried. When an Item Named Event is received, this internal list is updated, and when a Shop Items Projection is requested, the Name is injected. The problems with this include maintaining two models (shop items and item names), changing from a straight query->response model, to query->join->response model, and duplication of the Item Name list, as other systems will probably maintain their own Item Name lists if they need that information. This problem is further worsened if, in addition to Item Name, we need a Sprite/Image/Picture, Description, or anything else.
4. Have a separate, out of context/domain Item Name list, and use its Domain Service to query the correct Item Name. This Projection wouldn't have to maintain the list, the list wouldn't have to be duplicated by other systems, but to query then involves querying something else too. Problems with this include querying a separate Read Model when the Shop Items Projection is queried, moving to a query->join->response model like above, and it feels like it breaks down the CQRS boundary a bit.

Core aspects that influence these solutions are:

1. Is it a design smell to maintain multiple separate Read Models, even for one use-case?
2. I noticed there was kind of a separation of concerns in the Read Model itself. One side is the querying of the Read Model (query->response vs query->join->response), and the other side is the updating of the Read Model (iterate through all, even if it won't be updated vs iterate only through ones we know we're going to update).
3. Cross context/domain querying seems to go against the grain of CQRS. It feels awkward to try to query something from another domain/context. It feels like this projection should have absolutely all the information it needs for this use-case. If it is missing something that it needs, it seems like an indicator that there are events that it's not listening to that it should be listening to.

Answer 1

Before taking the question on. It would be worth going in to the overview of CQRS

What is CQRS?

• As you said CQRS is about separating the write model and read model

Some benefits of CQRS

• The reading logic of the application could have quite some complexity to show different types reports. Splitting the models out makes the write model simpler to implement and improves the code readability drastically

• The write model generally won't need complex querying capabilities, so we could use databases that are more optimized for reading based on primary key (or something like that) such as EventStore, Key Value store, Table store, Document store, etc...

• In the read model, there could be various reports. And we could choose a database and also a schema to provide us the desired performance.

Consideration for choosing the persistence store for Read Model

• The database and the schema has to be chosen for the read model so that we get a good read performance for the queries

• When we choose to implement a CQRS pattern, we are settling for eventual consistency. So it would be certainly okay if it take some bit of time for ingesting events. However if the ingestion time is too much it will not be acceptable as it will take too long for reports to reflect the current state.

Suggestion

1. Architecturally, it would be a certainly a good idea to separate the business logic from the data base layer. However in the read model, it's mostly about data access. So when you separate this layer, my suggestion would be provide implementation to you client for few databases
2. If RDBMS is the choice, storing the json and storeid in a table would be a sub optimal way. Your ingestion of events is going to take quite some time as it involves deserialization. However your reads are going to be fast. My strong suggestion would be change the schema or change the type of database

Change suggested

1. If you use a RDBMS, either have a fully normalized or partially normalized schema. Having serialized information in the database is going to make your updates expensive in terms of time. when you have normalized tables, you could have indexed views to make your queries faster. You could still improve the query speed by an cache such as Redis if need be
2. A document DB such as mongo DB suits your scenario best. MongoDB is going have documents stored in json form. You can filter and update documents that have a specific product. So this will make your ingestion and reads fast

Answer 2

It seems like solution #1 is the way to go for what (I think) you want, with solution #2 being a possible performance optimization you could add later, if necessary.

It sounds like you'll already have a list of the shops, so that part isn't a problem. Even if each shop's data is stored in its own text file, I don't think it will take all that long to parse them and update as needed, even with a few thousand shops (and if they're in an RDBMS, practically no time at all).

Unless you're rebuilding the entire game's shop data from scratch, you'll only need to apply the rename "search and update" process one time for each update. So unless these items are updating often, or you have hundreds of thousands of shops (not in an RDBMS), you'll probably be fine.

If the updates are frequent or you do have tons of shops, then maintaining a shop<->item list will likely be worth it for more efficient updating. You could store the list in memory and rebuild it when the server restarts, persist to a file/DB, or some combination. (Internal read models that are just there to help the system do its job are 100% valid!)

Microservice Approach

If you think of it like a microservice architecture, you have the game that's in charge of defining what the items are, and the shops that are using the items from the game but are in charge of their own inventory.

In that case, you wouldn't want each shop to have to ask the game for the list on every operation, so you would have each shop maintain its own copy of the list (like you have now). Then each shop would update its list with changes from the game service (on a schedule or by subscribing to events).

The difference here is that the shops aren't independent services, but are part of the game. So when the game has updates, it goes out and updates each shop (as opposed to the shop listening for events and updating itself).

In any case, I think option #1 would be the best bet, and should work regardless of the persistence method chosen (some will be better at scaling up than others, but all should work).