0

I have question on how to design an order management microservice. An order has different workflow based on the previous state or based on the business needs. I'm guessing at some point cannot keep on adding if/else/switch because it would be very difficult manage workflows and this will also leads to more bugs.

Order Management service has to manage the order/items in the order from creation to delivery. At each every point there could be sub workflows like Rewards, Cancellation etc..

Moreover, most of the order management work is done event-driven ( Ex: if the change happens then listen and trigger another action ) & there are some ad-hoc workflows which requires to backtrack order to previous states or move/skip some actions altogether and continue processing different workflows.

What I have already Tried:

  1. Kind of StateMachine based workflow management.

Pros:

  1. I was able to come up with approach to manage different workflows by creating JSON DSL based on configs and select and continue processing the workflow based on that config.
  2. Able to manage 10+ different types of orders now.

Cons:

  1. Number of config files are continuing to increase.
  2. Not able rearrange or change the workflow at runtime.
  3. Changing the order of workflow affects the existing orders placed.

My question are,

  1. How do really big companies like Amazon/Shopify maintaining the workflow ?
  2. What design pattern we have to follow to make sure we are able to scale ?
  3. What design pattern we have to follow to achieve observability ?
2
  • Sharing your research helps everyone. Tell us what you've tried and why it didn't meet your needs. This demonstrates that you've taken the time to try to help yourself, it saves us from reiterating obvious answers, and most of all it helps you get a more specific and relevant answer. Also see How to Ask – gnat Jun 11 at 12:09
  • 1
    Added what i have tried now @gnat – Sathish Jun 11 at 12:14
1

A common solution in big companies is to create a state machine for each order. The implementation might change, sometimes the state machine is implemented by a BPM engine, sometimes by a rules engine, sometimes it is implemented using plain code. But the logic is always the same. BPM and Rules engines have also some open source implementation which may include an audit mechanism. But basically you need a key or rule based logic to match the right event to the right order and probably a a caching mechanism to retrieve an order when a related event is triggered (An order might take few hours or several months to be fulfilled). Then how you implement the behaviour for each state depends on the type of implementation you choose.

2
  • Thanks for the information. Plain code doesn't seems to scale well. I have implemented a kind of state-machine framework which has json based list of actions defined and each and every action decide what is the next action to execute :) guess state machine is also not a best way to go. BPM Engine and Rules engine seems to be something i can check out. – Sathish Jun 11 at 12:32
  • 1
    @Sathish In this case keep in mind that BPMN engines give you a nice graphic overview and a lot builtin features, but the design will be less flexible. While rules engines will give you more development to do and will be more difficult to scale, but they are a lot more flexible. – FluidCode Jun 11 at 12:44
1

One approach that works very well with transaction based interactions (like order management, supply chain management, stock trading, etc) is Event Sourcing. With an Event Sourced architecture, your event stream is your model. You then can project that stream of events into the forms that better meet your query and display needs.

In this approach, every order will have its own Event Stream (list of events from the beginning of the order to its completion).

This architecture works well with Command Query Responsibility Separation (CQRS) where Commands perform all the validations and preparatory work needed for your business process, but the end result is appending a message to the Event Stream for your Order.

Naturally, the concept of an Event Store has come up to derive the need of persistence. The Event Store will store the chronological stream of events for each Event Stream so they can be replayed.

If you combine the two you have a very powerful architecture. You can have separate microservices for the business logic side of things and the display side of things. The business logic is done in Commands, and writes events to the appropriate Event Stream into the Event Store. The view side of things reads from the Event Store the events not processed in the Event Stream, and Projects it into a database (RDBMS, graph, document, search) that is better suited for querying. That allows you to scale the writes and views separately if that's needed.

If necessary, you can build new Command microservices for each of the new responsibilities. Those commands perform the validations and additional logic needed to make sure the process is done correctly, and the end result is an Event in the EventStream to record that the work has been done.

Critical aspects of Event Sourcing:

  • Events are in Past Tense, indicating work that has already been completed
  • Projections (in functional languages, a left fold) transform the Event Stream for display or query purposes
  • The Event Stream is your authoritative state

Distinction between Event Sourcing and Event Driving:

  • Event Driven architecture has events in Present Tense, and reflect commands that will be handled by another service
  • Event Sourced architecture has events in Past Tense, reflecting that the work has already been done (AKA facts)
  • Event Sourcing does not require sending events over a queue, a similar effect can be had by having other services query a shared event store

Pros:

  • Work can be distributed among many microservices that handle one function, allowing you to distribute logic appropriately
  • The Event Stream doubles as an authoritative audit log

Cons:

  • Event Versioning is something that you have to consider carefully
  • There can be conflicting events, requiring human review--particularly in a distributed environment

NOTE: CQRS is not necessary with Event Sourcing, but it fits naturally and works well with it. Many examples you find with Event Sourcing is coupled with CQRS.

2
  • Wow, Thank you so much for the detailed explanation. I really have to read this answer thrice to understand how much info you have provided. But i do have lot of question with respect to your answer. Before asking question i want to start learning CQRS and Event sourcing :) – Sathish Jun 11 at 13:56
  • 1
    The video I linked to provides a good introduction to the concepts along with a working example. – Berin Loritsch Jun 11 at 15:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.