Best practice to architect a system in which local users may override global definitions

Question

Our travel industry client, operating across three continents and several countries, wants to standardize the process of choosing travel packages, air routes, hotels, and hundreds of other travel options. Today, local travel agencies decide themselves what they want to offer or not. The idea is to let them make decisions still, but with some constraints coming from global marketing.

Technically speaking, we need the following:

1. Three level of players - Global, Continent, and Country.
2. The process starts with Global, who will define which products the whole enterprise can work with.
3. After definition by Global, Continent will be able to refine it, but always inside the boundaries defined by Global.
4. After refinement by Continent, the same will apply for Country. They can refine even more what was defined by Continent, but again inside the boundaries coming from Global and Continent.
5. The system must then be able to support hundreds of definitions made by Global - one definition per attribute (air travel, hotel, ship, etc.). For each definition made by Global, we can have up to three definitions made by Continent (one per continent) that are derived from Global and inside each continent (up to n definitions made by the n countries from that continent).
6. If Global edits an already existing definition, it must be communicated to continents and countries. That means dozens of definitions need to be notified and changed. And this is where I am stuck.

A simple example to visualize the business requirements:

• Global defines that hotel A, B, C, and D are available.
• Continent then says that they want to work only with A, B, and C because D has a bad reputation inside the continent.
• Country then says that they want to narrow down the list more, they are willing to work only with A and B, because C does not follow the religious rules of the country.

In this very simple case, we need to hold three definitions for the same domain (one for Global, one for Continent, and finally one for Country).

Question

What is the best way to communicate to local users that a change has been made by Global or Continent?

Assumption: I will create only one service to handle this requirement.

I was considering using queues, however the service will then be both producer and consumer, which seems a bit odd to me. Has anyone faced a similar challenge and found a satisfactory solution?

Answer 1

If all we're doing is customizing an available products list this is doable.

• Global provides a white list of products
• Continent provides a black list of products
• Country provides a black list of products

If global edits an already existing definition, it must be communicated to continents and countries. That means dozens of definitions need to be notified and changed. -> This is where I am stuck.

When Global adds a new product do you want it to come in automatically or not? If you want automatic this is fine as is. Otherwise you need to give continent and country their own copy of the whitelist that they can update to the new one only after reviewing the changes and making their own changes to their black lists.

_{The inherit assumption I'm making here is that the black lists will stay much shorter than the white list. If the black lists become significant it may become worth it to partition the white list into regions.}

You will probably want to give them some tools to make it easy to notice changes to the white list.

I'm sure you have other issues and business rules that this might not handle well. But this is the simplest solution I can think of for what you described. It might be a good starting point.

Also, consider making the conjugation of the names of the definers consistent:

Either

Globe, Continent, & Country

or

Global, Continental, & National

Answer 2

I think candied_orange's answer is a good start. However, for a real world system, you may need to extend the concept:

• Global provides a list of products together with a "default" boolean setting for each product ("use" or "don't use")

• Continent keeps a list (subset) of all products where the default setting is not inherited from global, but with its own setting. So either Continent changes the setting to the opposite of "use" or "don't use", or fixates it, so it won't change when the default setting in global changes afterwards. It may also define products which are stripped from the global list and hence not visible to any country of the continent. Each product which isn't part of those lists inherits the setting from Global

• Country does the same - define individual settings for a product subset from the Continent level. For products not in the subset, the default setting stays valid.

Now, when Global's product list gets extended, or the default setting of an existing product is changed, the new default setting for the product will automatically be passed downstream the hierarchy.

Of course, from time to time you may have to inform the people maintaining continent and country settings that they double check if the default setting for the new product is ok for them. But a well chosen default setting may free all continents and country managers from reacting immediately for each and every change to the global product list.

This may be more effort to manage than just simple white lists and black lists, but will allow a lot more complex scenarios. Moreover, it can be extended to more properties of different types than just a boolean "use" or "don't use".

Answer 3

You need to consider how you are going to model the various constraints and how you will determine the most restrictive constraints for a given geography and client. Everything else is secondary.

One option is to maintain many constraint sets: one at the global level, one per region, and one per customer.

class Constraints:
  permissible_hotels: Set[Hotel]

To determine the most restrictive constraints for your three tiers, you can write some code that does a pairwise calculation and run it two times. First with global and region, then apply that product against the customer.

class Constraints:
  ...

  def restrict(self, other: "Constraint") -> "Constraint:
    restricted_hotels = _determine_narrowest_hotels(self.permissible_hotels, other.permissible_hotels)

    return Constraints(permissible_hotels=restricted_hotels)

If you need to notify customers when constraints are changed at any of the three tiers then you need to represent the global, regional, and customer entities in a directed object graph.

class TripPlan:
  constraints: Constraints
  id: uuidv4
  downstream_trip_plan_ids: Set[uuid4] = set()
  interested_party_email_addrs: Set[str] = set()
  parent_id: Optional[uuid4] = None

  def update_constraints_and_notify(self, c: Constraints) -> None:
    self._persistence_adapter.upsert_trip_plan(self)
    self._notify_interested_parties_recur()

  def _notify_interested_parties_recur(self) -> None:
    self._email_adapter.send_constraints_updated_email(self)
    
    for id in self.downstream_trip_plan_ids:
      tp = self._trip_plan_adapter.find_trip_plan_by_id(id)
      tp._notify_interested_parties_recur()

For a small object graph with just a few customers and a limited number of constraints, you can just store the entire domain in a flat file, if you wanted, and load the object graph in memory at init. In that case downstream_trip_plan_ids can be downstream_trip_plans and you of course don't have to actively retrieve each vertex when you're doing the notification process.

For any given TripPlan, you can follow its parent chain to calculate the most restrictive set of constraints.

class TripPlan:
  ...

  def calculate_restricted_constraints(self) -> Constraints:
    if not self.parent_id:
      return self.constraints

    parent = self._trip_plan_adapter.find_trip_plan_by_id(self.parent_id)
    predecessor_restricted_constraints = parent.calculate_restricted_constraints()

    return self.constraints.restrict(predecessor_restricted_constraints)
  

By the way, if you model the problem with this approach then you could add additional tiers without modifying the algorithm or data structures, if the need should arise.

---

I was considering using queues, however the service will be then both producer and consumer, which seems a bit odd to me.

Your domain is self-referential (each tier depends on the one above, and the "stuff" stored at each tier is basically the same). So it's only natural that any code modeling the tiers, traversing them, or persisting them is going to be both a producer and a consumer, in a sense.

(And as general architectural advice, it's fine if a queue's consumer has to publish messages back to the same queue as part of its duties; you just have to write very good tests to make sure you don't create an infinite loop.)

Answer 4

I've made a few of these types of things and invariably users fail to understand how the hierarchy works and end up defining everything at the lowest level with loads of duplication.

Any change to the higher levels the comes with a massive risk of unexpected results. "I deleted the global rule for hotel A, but i didn't think Superspecial customer C would be affected!" etc

So I would stick with your initial approach, each travel agency gets a full rule set for each country.

Keep your structured rules for generating defaults for new agencies.

When you change the structured rules, generate the new rule set for each agency, then generate a difference list between that set and their existing set.

Allow a manual checking stage where users can review the changes, undo and redo them multiple times.

Send out the difference list with explanation. "We have had to remove hotel A from the global availability list (not for you super special customer!) Here is the changes to your ruleset, please check and let us know about any issues. We will apply these changes on {date}"

Manually tweak the changes based on human responses, then apply each set.

Answer 5

I would work with change-sets and a separate dataset per player (Global, Continent, and Country).

So when a Continent is initiated it gets it's first copy of the dataset from Global.

Then a Country gets it's first copy of the dataset from Continent.

From that moment on you generate change-sets to push data down, for example:

UPDATE hotels SET available = false WHERE hotelId = 123

Preview

You then give the user who is pushing down a preview:

SELECT country, available FROM hotels WHERE hotelId = 123

For example in a format like:

US: 1 hotel will be disabled
UK: 1 hotel will be disabled (was already disabled)
NL: 1 hotel will be disabled

You can do this in SQL, JSON objects or whatever you want. Format is not relevant.

You can keep all hotel records in one table. I would move the meta data about the hotels into a separate store (hotel_info) (which is universally equal). So that one contains the name of the hotel, description, link to website etc.

Country specific changes

Then if something needs to become country specific you move the hotel_info field to the hotels table. For example if the reservation link becomes custom per country:

Change-set:

ALTER TABLE hotels ADD reservation_link

UPDATE hotels SET reservation_link = "https://hotel.example" WHERE hotelId = 123

ALTER TABLE hotel_info DROP reservation_link

You then update your system to use this link field (still works because the field is pre-populated with the universal value so the data for the users stays the same) and now the locals are able to modify it:

UPDATE hotels SET reservation_link = "https://hotel.example/UK-RESERVATION" WHERE country='UK' AND hotelId = 123

This works the same from Global to Continent. Exactly same process.

Overrides

This way you can keep your override also clear, for example you could let a local set:

UPDATE hotels SET available_override = true WHERE country='UK' AND hotelId = 123

If you allow them to override decisions from global for example.

Overview

It is easy to monitor where local changes are made: Example: Hotel is not available globally:

SELECT hotels WHERE available_override = true WHERE country='UK' AND hotelId = 123

The you get a list of the local non-conforming countries. You can make this logic as simple or complex as you want. For example if a hotel is burned down your changeset can override the local overrides with a simple update.

Notification

To notify the users I would store the change-set itself anyway in a table for debugging purposes. Those records can also serve as your notifications log. You could store in the change-sets table even whether is has been accepted/taken care of/reviewed for example.

Views

If you need a combined result (for example for the website) you can join or create a view to have a country specific table available. Or you could cache the hotels_info as duplicate cache in the hotels table.

Structure

This gives you a generic structure which allows you build upon, makes clear what the changes will do and is extendible in the future when further requests come in to customize per country.

Also it is fully auditable which changes have been made, by who etc.

Also it gives you the hotels table where you can store country specific data unrelated to global. Like a "Advised hotel top 10" where the data in the field is locally defined and only relevant locally.

Answer 6

I helped architect a POS and inventory management solution for a nation wide company that executed growth through local acquisitions and re-branding. Because of this approach, they required a rules system not unlike what you've described, just with a different domain and additional complexity. One of the approaches we utilized (with success) was to have just a single 'Rules' table that would return a composite value that not only 'answered' the question of availability, but also gave the end user (employee) the ability to see why a restriction was in place, override if needed, and also (most importantly) separated the rules from persistence, so that an update to the rules would not only not invalidate existing reservations, but also let us 'replay' scenarios for audits e.g. if we needed to go back and find out why a client could/could not make a specific reservation.

The table in your case, might look like this

Create table Rules (
    ProductId       int,  //FK to a products table
    Priority        enum  // Application Order for this rule
                          //  e.g. Global=1, Continent=5, Local=10
    Restriction     enum  // Application of this rule should result in...
                          // e.g. Available=1, Restricted=2, Prohibited=4,
    ActiveDateStart Date, //Date the rule should be active
    ActiveDateEnd   Date, //Date the rule should end
    Description     text  //Reason for this rule, additional info
)

This lets you query only active rules and then apply them in any order to determine whether a product should or should not be available by masking the Restriction value in the Priority order. This is the sort of logic that you might see in a CRUD permissions system and allows you to check for complex scenarios like 'available but prohibited'. This is important because some products may not exist at a global level and some restrictions might be invalidated only at a local level (meaning despite a global restriction, there is an exemption in one region b/c reasons). You also don't need to query across services and look ups can be quick since queries can be constrained by active dates and products. Lastly, to your needs, notifications can be triggered by the activation of a given rule and using the dates, pre-emptively setup and sent out to your clients/employees so they can be aware of upcoming issues that they may need to be apprised of.

Lastly, you also gain the ability to show an employee the full set of rules being applied so that they can override if needed (restricted application) but still understand the consequences of what they're overriding.

In any case, there are some other interesting and complex answers in here, so it's just as likely I've missed something before I've had my morning coffee...