State machine - how to handle outside environment values?

Question

I've got a state machine implementation in Unity that I'm working on (C#), the plan being that it will be mostly used for AI related things.

I'm not sure how I should deal with various "inputs" / how it should interact with knowledge from the outside environment. Two approaches I've considered and tried so far:

1, I have a dedicated "Query" class that holds various bools. At the end of the Tick() method in each state I make some checks like

if (queries.JumpUp) { SetState(JumpState); }

that take care of switching states. To change states, I simply set the bool to true. This seems to work fine, creates a very loose relationship between the "query" and the resulting behaviour, and lets me place pretty much all the transition logic in a dedicated method (my base Tick() method calls an CheckForTransitions() method at its end, and it's this method that I override and put all the transition logic in). It works fine so far, but I'm a bit worried that this type of logic might be almost a bit too loose. I think it might be somewhat similar to the blackboard design pattern. It also feels a bit like an observer pattern, which could be useful - I might have multiple different types of state machines active at the same time. A "Query" class that has booleans like I mentioned seems like a very natural way of implementing a general interface layer that will allow for loose "communication" between the layers etc.

2, Create virtual methods for all possible "events" I would like to be handled.

public override void TryJump() { SetState(JumpState); }

To change states, I explicitly call the method above (or some wrapper around it, implemented inside the StateMachine).

This also seems to work fine. Some slight negatives I can see compared to 1: With approach 1, there's no issues if I choose to update my state machine independently of the game loop. With approach 2, there could be multiple calls that result in a transition between two state machine ticks/updates. This could be fixed by the making calls like "TryJump()" change some buffer variable that would just hold the desired next state, and the actual transition would perhaps happen at the end/start of the state machine update (rather than "TryJump()" causing an immediate change in states). But at that point I'm getting very close to the approach used in 1, so I'd think I might as well just use that.

I don't have a single nice place in the code where I can check exactly what sort transitions can happen and what their conditions are. I will have to have tons of virtual methods for each state, for each "TryJump" type event. If I don't want to work with a direct reference to the state machine, I will have to also write wrappers, that will call those TryJump events the currently active StateMachines - so something like say:

public void TryJump() 
{ 
 foreach (StateMachine stateMachine in ActiveStateMachines) 
     stateMachine.currentState.TryJump(); 
}

Despite that, something just feels a bit off about the method in 1, - using booleans like that just seems a bit weird. I'm also not terribly comfortable with event based approaches, and it feels a bit "wrong" to use a special layer with booleans etc., when I could just make a direct call that does what I want using approach 2.

EDIT - adding some example code to clarify how I'm doing things so far.

class StateMachine
{
    private State currentState;
    public void SetState(SimpleState newState)
    {        
        state = newState;        
    }  
    public void Tick()
    {
        state.Tick();
    }
}

abstract class State
{
    public parentStateMachine;
    protected void SetState(State newState) {parentStateMachine.SetState(newState);}        
    public void Tick()
    {    
        Update();
        Transition();
    }  
    protected virtual void Update(){}
    protected virtual void Transition(){}
}

the state machine tick is then called for example as:

public void Update() // game loop update inside Unity3D
{
   stateMachine.Tick();
   queries.Clear(); // Clears all the bool values inside queries
}

an example of how a Transition override could look when using approach 1:

protected override void Transition()
{
    if (queries.PickupAProp && currentProp == null)
        {
            SetState(PickupPropState);
            return;
        }
    if (2secondsPassedSinceEnteringThisState() && queries.WalkToTarget)
        {
            SetState(WalkState);
            return;
        }
}

in other words, all the logic concerning decision of what state to choose next is implemented mostly in the state extension itself. "queries" really just work as something that holds various data / signals. It shouldn't contain any complicated logic and in my current implementation it's mostly booleans.

The alternative approach 2, could then replace certain "checks" like so:

public override void TryPickupProp()
{
    if (currentProp == null)
    {
        SetState(PickupPropState);
        return;
    }
}

In places where I did queries.PickupAProp = true; I could now instead just do stateMachine.state.TryPickupProp();

But Transition() should still contain the check:

if (2secondsPassedSinceEnteringThisState() && queries.WalkToTarget)
{
    SetState(WalkState);
    return;
}

As that's a purely "internal" check. But the TryPickupProp() query could now just be called directly.

Answer 1

What’s the problem?

Three actions are needed to manage the state transition:

1. detecting the causes that trigger a state change;
2. determining the target state;
3. switch from the current to the target state.

Unfortunately, your narrative does not tell how you keep track of the current state, which influences all three actions. But it seems that:

1. Your tick() function is related to the frame refreshment and triggers the detecting of state change.

2. Your two alternatives are about detecting the changes and determining the next state:

   • the first gives the responsibility detecting and target determination to the query class.
   • the second gives the responsibility of the detecting and the target selection to each state.

3. The state machine itself then makes the switch happen.

Your approach has in any case the inconvenience of an active poll for state changes; polling is always a waste of resources, especially if the state changes are not related to the ticking. Moreover, the polling requires to know what to poll for, and this creates some tighter coupling.

How to improve it?

Both alternatives might currently lead to a maintenance nightmare. So my advice is to brainstorm on the consequences of adding a new state or changing the state transition rules on your current code. Up to you to see how this could be dealt with, but this thinking could be the driver for a more robust design.

One approach would be to use a “queue” of events. Whenever something implies a state change, put a state change event in the queue. When tick() is performed, just check if there is a state change event in the queue and activate the target state. If delaying to the tick is not desired, just activate the change immediately.

Now, a usual debate is whether to externalize the state transition rules (e.g. in a table) or to let the state decide what comes next. In games, where states are rich, the second approach seems a good candidate. But unlike your second alternative, the state would not call an overridden virtual function directly: it would just put an event corresponding to the next state in the queue.

A last point is how to detect the events. It makes no sense to answer this, as it depends heavily on how you manage your current state and react to input, and we don’t know. But I think that once you’ve started to implement the other ideas, this part will appear straightforward.