Is there an approach to keep a large number of conditionals maintainable

Question

I work on a survey that has a lot of questions. This means we have a lot of columns/variables to work with. This translates to a lot of conditionals that have to be done a certain way according to a specification. The implementation in the code can be different, but it has to map to the spec. We are moving toward Python, and I've tried to think of ways to make the spaghetti code easier to wrangle and change when needed. At the same time, I need to point to the spec and say here it is in the code.

I've considered the strategy patterm or "chain of responsibility", and I was introduced to the specification pattern. But anything that requires creating a class for a condition or for a type of person, for example, feels like it won't work. I feel like it would result in unmanageable complexity. It's already tough because the data model is complex, and has so many columns to work with.

Current code can have points where we have if statements nested 10 deep. To me that's really tough to look at and make changes to, but a lot of people find that easier than what I like to do.

One solution I'm considering is to create a function for each check in the spec.

For example, I can give a simple example of a piece of spec.

Check1 (Age < 18) -True-> 
Check2 (Sex = Male) -True -> 
Set description = "Male Under 18"

One possibility is to flatten that arrow anti-pattern, which speeds up the Python program. For example:

universe = (interviewed_person == "yes")
check1 = (age < 18)
check2 = (sex = "Male")
if (universe & check1 & check2):
    set description = "Male Under 18"

Functions would allow us to do unit testing for each check and more easily track program edit flow. I had a mistake in my real example that I found when I did this way. With these complicated specs, I know there's only one or two ways to get to a certain point in the spec. The only way to get to Check1 is if the person is in universe. The only way to get to Check2 is if Check1 is true. The below program is an example of that, just imagine if it's 200 methods x 200 edits. The below prints the following to the terminal or command line: stop True Male Under 18

class Person:
    def __init__(self):
        self.job= -9
        self.age = 17
        self.sex = "Male"
        self.interviewed_person = "yes" #for the example, let's say everyone's in universe

class Logic:
    def __init__(self):
        #track state value with another class?
        state, condition = self.universe(p.interviewed_person)
        state, condition = self.check1(state, condition)
        state, condition = self.check2(state, condition)
        state, condition = self.set_description(state, condition)
        print(state, condition, p.job)

    def stop(self):
        return "stop", True

    def universe(self, state):
        if (state == "yes"):
            return "universe", True

    #putting else in here to show that we may have False conditions that lead elsewhere
    def check1(self, state, condition):
        if ((state == "universe") & (condition == True)):
            if (p.age < 18):
                return "check1", True
            else:
                return "check1", False

    def check2(self, state, condition):
        if ((state == "check1") & (condition == True)):
            if (p.sex == "Male"):
                return "check2", True
            else:
                return "check2", False

    def set_description(self, state, condition):
        if ((state == "check2") & (condition == True)):
            #I've worked this with my data in objects, but I may go with dataframes
            p.job="Male Under 18" 
            return self.stop()
        return state, condition
    
if __name__ == "__main__":
    p = Person()
    l = Logic()

I didn't find the following link helpful for my situation: Style for control flow with validation checks

I think the specification pattern would end up with too many classes and be unmanageable. If I have hundreds of checks, like Check1 - Check200 times 200, I think it would add a lot of lines of code. Is there another way?

Answer 1

Is there an approach to keep a large number of conditionals maintainable?

Here's two:

• It's ok to use Boolean expressions to set Boolean variables
• Early returns are ok in languages that have finally blocks

While were at it:

• Stop making initialization interesting. Set your fields, validate, and get out of the way.
• Mutating fields is fine when on a single thread

Here are a few rewrites to make these points:

class SpecificationLogic:
    def __init__(self, person):
        self.person = person
        self.state = person.interviewed_person
        self.condition = True

    def universe(self):
        if (self.state == "yes"):
            self.state = "universe"
            self.condition = True

    def stop(self):
        self.state = "stop"
        self.condition = True

#removing else in here to show that variables are our friends
#also, don't use the bitwise & unless you really mean it
    def check1(self):
        if ((self.state == "universe") and (self.condition == True)):
            self.state = "check1"
            self.condition = self.person.age < 18

    def check2(self):
        if ((self.state == "check1") and (self.condition == True)):
            self.state = "check2"
            self.condition = self.person.sex == "Male"

    def set_description(self):
        if ((self.state == "check2") and (self.condition == True)):
            self.person.job="Male Under 18" 


    def myPrint(self):
        self.checks()
        print(self.state, self.condition, self.person.job)

    def checks(self):
        self.universe()
        self.check1()
        self.check2()
        self.set_description()
        self.stop()            

if __name__ == "__main__":
    p = Person(job=-9, age=17, sex="Male", interviewed_person="yes") 
    s = SimpleLogic(p)
    s.myPrint()

Here I've removed some needless if structures to show off how handy nicely named Boolean variables can be. This is meant to be a refactoring so the behavior shouldn't have changed.

You've done a fine job of ensuring these methods are called in order and protected them against calls after a check fails. So much so that I question the need for all these methods. This only makes sense to me if, for some reason, you need the ability to stop between calls and do something else. If that's just crazy talk then why not just do this?

class SpecificationLogic:
    def __init__(self, person):
        self.person = person
        self.state = person.interviewed_person
        self.condition = True

    def myPrint:
        self.checks()
        print(self.state, self.condition, self.person.job)

    def checks(self):
        self.condition = self.state == "yes"
        if (not self.condition): 
            return
        self.state = "universe"

        self.state = "check1"
        self.condition = self.person.age < 18
        if (not self.condition): 
            return

        self.state = "check2"
        self.condition = self.person.sex == "Male"
        if (not self.condition): 
            return

        self.state = "stop"
        self.condition = True
        self.person.job = "Male Under 18"

Just like the other solutions, this will report how far it got (see state) before it fell off the happy path.

Will these patterns solve everything? No, but keep them on your tool belt. It's worth working very hard to make something simple and easy.

^{Forgive me. I haven't tested this code. Use with caution.}

Answer 2

        self.job= -9
        self.age = 17

The age makes sense, number of years alive. For job, consider adopting enum or one of its several competitors.

        self.sex = "Male"
        self.interviewed_person = "yes" #for the example, let's say everyone's in universe

For interviewed status, maybe a boolean? Or again, enum, as with gender.

Within Person I was kind of hoping to find @property def description ....

The proposed Person class is a noun, and it makes perfect sense.

I confess I do not understand what a Logic is. It sounds like it wants to be a business logic, but it seems to be more about verb evaluation than about nouns. Maybe there is some Business Process which it models, and which has complex associated logic?

The semantics of (state, condition) are unclear. Apparently a Person would completely specify them?

---

Consider representing these various demographic buckets in terms of predicate functions, and then use

$ pytest --cov

to measure line coverage of each function. The goal would be to surface logical clauses, perhaps 9 or 10 levels down, not yet exercised by the data. Armed with such coverage measurements, you would be in a good position to augment the data or to construct new helpful unit tests.

---

            return self.stop()

Now that is interesting! Perhaps what you're looking for is to raise StopIteration or otherwise signal that we're at a terminal node?

---

Overall, the goal is unclear to me.

Perhaps one way to phrase it is we have many observations, and we wish to bucket them, with as much specificity as possible. So we might have a state machine which we advance through as many states as possible until it reports "Stop".

Here is a completely different perspective. Consider all observations, and their K features. Find the feature that will most evenly partition the set into K-1 features. So initially we might find a nearly 50-50 split on gender. And then do that recursively, while smallest bucket is "too big". At the end we have meaningful labels describing large swaths of the entire dataset.

Answer 3

Unfamiliar with Python so please excuse any syntax butchering. One way to tackle this is to recognize that you have inputs, conditions, and questions

Inputs are the attributes like age, gender, etc. One or more answers to previous questions may be attributes as well.

Conditions are boolean clauses like (age >= 18) or (gender == "Male") or (answer_54 == "lemons"). When all of your conditions are true, the question is valid and should be asked/shown.

Whether a question is asked/shown is determined by a (finite) number of conditions. One can work backward from the conditions to determine the inputs needed to ask/show a question.

One method per question, taking the needed inputs and returning true/false will enable you to encapsulate all the logic for each question in one place. The number of methods will scale linearly (with the number of questions). It will be trivial to find (and test) the logic for any given question.

This gets a little more complicated when dealing with hundreds of inputs. This may lead us to create a class or use a dictionary to store them and pass them into each "question method." In this case, each such method should validate that all needed values are present in the passed-in dictionary as a first step.

Moving from method per question to class per question may make this easier to come to grips with. This also offers the change to create an Interface with methods that are called before or after the question is asked.

Sounds like a fun challenge. Flattening the logic is definitely the right direction to head. 10+ levels of nesting is a maintenance nightmare waiting to happen. Good luck!