How to avoid messy variable dependencies for ease of testing in Node?

Question

I couldn't decide if the question belongs on Stack Overflow, but I went with SE Stack Exchange for the higher quality, broader answers. Bare with me.

Examples are in Node but I guess applies to a multitude of languages

I'm big on writing pure code without side effects. It's easy to reason about and to test. Yet, the following situation has annoyed me for years, and I need a resolution, as I can't quite solve it definitively.

For a Node module (really any self-contained unit of code), I always envision something like this:

const yourThing = {
  doYourThing: (params) => {
    // A bunch of function calls, all pure,
    // All beautifully harmonious
  }
}

const fun1 = (params) => ...
const fun2 = (params) => ...
const fun3 = (params) => ...

module.exports = yourThing

But it always devolves into...

const const1 = 0
const const2 = 1
const someLoadBearingBool = false;

let ChangingVariable1;
let ChangingVariable2;
let ChangingVariable3;

// big and complex
let someState;


const yourThing = {
  doYourThing: (params) => {
    // Assign changing variables
    someState = { param1, ...};


    dirtyfun()
  },
  ...
}
// All these are changing the module-global state;
const dirtyfun1 = (params) => ...
const dirtyfun2 = (params) => ...
const dirtyfun3 = (params) => ...

module.exports = yourThing

The dirty functions are not easy to test. Sometimes the order in which you run them matters, as they keep switching variable values on each other.

Sometimes I can't change the signature of these functions, as they must be so by contract, used elsewhere, so injections are not really viable, without major overhaul.

It makes it very annoying to write good tests, and hard to keep track of what's going on.

I see the shadow of this problem in almost all my code, it's almost like there's a bug in my thinking and I feel it seep into larger structures (like weird dependencies showing up across the entire software). Also it slows down my productivity, as I sometimes end up working waaay too hard/long to minimize this.

(I'd also like to avoid depending on libraries like rewire, as this introduces an extra dependency to your project, and one specifically to fix something that (in my mind) shouldn't be broke.)

I have yet to find a general solution to this.

What are some good techniques and methods to mitigate this to the greatest extent possible? How do you suppress your annoyance if this is unavoidable?

Answer 1

Perfect is the enemy of good.

Sorry but if you're trying to get something done you'll always be able to find some ideal principle that you're compromising to get something done.

Why? It's the law of diminishing returns. Put enough work towards any ideal and eventually it's costing you more than it will ever return.

Seek balance. Rather than demand all functions be perfectly pure, make clear which functions are pure. After all the argument is that pure functions are "easier to reason about". Sure when you know that it's pure. You see side effects even in the "pure functional" languages. You have to. So just tell me when you pulled it off somehow. And don't get it wrong.

Marking purity isn't a new idea. It's been proposed as a feature of new language. Because it's all to easy to drop into a pure function and start adding hidden side effects.

Without that kind of support all you have is discipline. So this becomes yet one more thing to sweat in a code review. And you have to ask yourself, every time, is it worth it to worry about this?

Keep your functions short.

The best advice I can give you to keep this from being a serious drag is keep all your functions short. That way a glance tells you which flavor of function you're looking at. This is more reasonable simply because purity isn't the only benefit you get from short functions. They're easier to read, low on complexity, and focused. This helps more than just the pure problem so it's easier for this to be worth it.

Separate pure code from impure code

slidetodoc.com - Computer Science 312 IO and Side Effects Functional

It's easier to reason about a pure function when you expect it to be pure. Keeping it where the other pure functions are helps you do that and helps keep it pure.

// All these are changing the module-global state;

Module-global makes me think this could have been a closure or an object. Now hold on. Don't freak out but objects and pure functional code can work together.

If you're going to let objects be part of this I highly recommend immutable objects that only take their state from a constructor once. Done like this they become much like closures. They just offer more than one thing.

That said, you still have closures. Dump state in a closure and you can stop thinking about the state and use it at different times. This decouples knowing the state and knowing the data you wish to act on.

There are also monads. They can manage side effects for you. I'll just point you to this explanation of them. You can build them yourself in most any language if you're crazy enough.

Of all of these the most reliable ideas are small functions and keep globals as local as you can. The rest, well, your mileage may vary.