Using state machines when managing remote resources

Question

So I’ve got a service I’m trying to clean up. It responds to requests by starting a stopped server on Amazon EC2, waiting for it to come online, then configures a 3rd party piece of software on the server through a rest api, and then sets a task to start on that 3rd party piece of software. The service also does some maintenance type stuff and tries to keep track of “orphaned” servers or servers that no longer need to be running and shut them down. It keeps state in a database currently.

Right now it’s kind of working like an informal state machine. The “task” is the unit that is being managed, and the task has all sorts of states that it can be in, but basically most of the states are just steps in the initialization process. The code is pretty messy, and adding features can be somewhat painful. For example I am struggling to find a way to cancel the task while the initialization process is underway without writing in manual checks to see if it is canceled at the beginning of each step.

I thought a state machine would be a good way to clean everything up and standardize the code somewhat, but there seems to be a problem with that solution when I consider it more carefully. In a state machine you are directly controlling the state, and define the transitions between states that should be possible. In this system though I am really using AWS API and this 3rd party software API to tell me what the state of things actually is. Regardless of what my service thinks the state should be, the real source of truth is outside of my control, and I have to transition into whatever state it tells me it is in and work from there. So it kind of seems like I would end up making a state machine just to break basically all of its rules and manually control the state anyway.

Is a state machine a bad fit for this? Is there another pattern that should work better? Or do I just need to slowly work on refactoring things to make them better.

Answer 1

A state machine may actually be fine to use, so long as you divorce the control of the state machine from the state machine itself and the definitions on how to transition from one state to another.

Suppose you created a state machine called StateMachine and gave it three states, State1, State2 and State3. You might have something like the following pseudo-pseudo-code:

class StateMachine{
    abstract class BaseState{}
    class State1 extends BaseState{}
    class State2 extends BaseState{}
    class State3 extends BaseState{}

    BaseState state;

    void moveToState2(){
        if(state is State1){
            //logic to get variables to pass into State2's ctor, possibly from State1 and/or from input parameters
            state = new State2();
        }
        else if(state is State3){
            //logic to get variables to pass into State2's ctor, possibly from State3 and/or from input parameters
            state = new State2();
        }
        //already in State2! Yay!
    }
    void moveToState3(){
        if(state is State2){
            //logic to get variables to pass into State3's ctor, possibly from State2 and/or from input parameters
            state = new State3();
        }
        else if(state is State1){
            //logic to get variables to pass into State3's ctor, possibly from State1 and/or from input parameters
            state = new State3();
        }
        //already in State3! Yay!
    }
    void moveToState1(){
        if(state is State3){
            //logic to get variables to pass into State1's ctor, possibly from State3 and/or from input parameters
            state = new State1();
        }
        else if(state is State2){
            //logic to get variables to pass into State1's ctor, possibly from State2 and/or from input parameters
            state = new State1();
        }
        //already in State1! Yay!
    }
}

Thus, the state machine maintains its state and all appropriate variables needed for each state and also knows how to transition from one state to another, but does not control its own flow; that is done from the outside. And if you can, perhaps add an error state and transition into that upon invalid state transition requests. Or log it, or whatever.

Basically, state machines need not decide their own flow. They tend to, but they can also just define the states and the transitions.