If nulls are evil, what should be used when a value can be meaningfully absent?

Question

This is one of the rules that are beaing repeated over and over and that perplex me.

Nulls are evil and should be avoided whenever possible.

But, but - from my naivety, let me scream - sometimes a value CAN be meaningfully absent!

Please let me ask this on an example that comes from this anti-pattern ridden horrible code I'm working on at the moment. This is, at its core, a multiplayer web turn based game, where both players' turns are run simultaneously (like in Pokemon, and oppositely to Chess).

After each turn the server broadcasts a list of updates to client side JS code. The Update class:

public class GameUpdate
{
    public Player player;
    // other stuff
}

This class is serialized to JSON and sent to connected players.

Most updates naturally have a Player associated with them - after all, it is necessary to know which player made which move this turn. However, some updates can't have a Player meaningfully associated with them. Example: The game has been force tied because the turn limit without actions have been exceeded. For such updates, I think, it is meaningful to have the player nulled.

Of course, I could "fix" this code by utilizing inheritance:

public class GameUpdate
{
    // stuff
}

public class GamePlayerUpdate : GameUpdate
{
    public Player player;
    // other stuff
}

However, I fail to see how this is of any improvement, for two reasons:

• The JS code will now simply receive an object without Player as a defined property, which is the same as if it was null since both cases require checking if the value is present or absent;
• Should another nullable field be added to the GameUpdate class, I would have to be able to use multiple inheritance to continue with this desing - but MI is evil in its own right (according to more experienced programmers) and more importantly, C# doesn't have it so I can't use it.

I have an inkling that this piece of this code is one of the very many where an experienced, good programmer would scream in horror. At the same time I can't see how, in this place, is this null hurting anything and what should be done instead.

Could you explain this issue to me?

Answer 1

Lots of things are better to return than null.

• An empty string ("")
• An empty collection
• An "optional" or "maybe" monad
• A function that quietly does nothing
• An object full of methods that quietly do nothing^{see Null Object Pattern}
• A meaningful value that rejects the incorrect premise of the question
  _{(which is what made you consider null in the first place)}

The trick is realizing when to do this. Null is really good at blowing up in your face but only when you dot off it without checking for it. It's not good at explaining why.

When you don't need to blow up, and you don't want to check for null, use one of these alternatives. It may seem weird to return a collection if it only ever has 0 or 1 elements but it's really good at letting you silently deal with missing values.

Monads sound fancy but here all they're doing is letting you make it obvious that the valid sizes for the collection are 0 and 1. If you have them, consider them.

Any of these do a better job of making your intent clear than null does. That's the most important difference.

It may help to understand why you are returning at all rather than simply throwing an exception. Exceptions are essentially a way to reject an assumption (they are not the only way). When you ask for the point where two lines intersect you are assuming they intersect. They might be parallel. Should you throw a "parallel lines" exception? Well you could but now you have to handle that exception elsewhere or let it halt the system.

If you'd rather stay where you are you can bake the rejection of that assumption into a kind of value that can express either results or rejections. It's not that weird. We do it in algebra all the time. The trick is to make that value meaningful so we can understand what happened.

If the returned value can express results and rejections it needs to be sent to code that can handle both. Polymorphism is really powerful to use here. Rather than simply trading null checks for isPresent() checks you can write code that behaves well in either case. Either by replacing empty values with default values or by silently doing nothing.

The problem with null is it can mean far too many things. So it just ends up destroying information.

---

In my favorite Null Object Pattern thought experiment I ask you to imagine a complex Rube Goldbergian machine that signals encoded messages using colored light by picking up colored light bulbs off a conveyor belt, screwing them into a socket, and powering them up. The signal is controlled by the different colors of the bulbs that are placed on the conveyor belt.

You've been asked to make this hideously expensive thing compliant with RFC3.14159 which states that there should be a marker between messages. The marker should be no light at all. It should last for exactly one pulse. The same amount of time that a single colored light normally shines.

You can't just leave spaces between the messages because the contraption sets off alarms and halts if it can't find a bulb to put in the socket.

Everyone familiar with this project shudders at the thought of touching the machinery or it's control code. It is not easy to change. What do you do? Well you can dive in and start breaking things. Maybe update this things hideous design. Yeah you could make it so much better. Or you could talk to the janitor and start collecting burned out bulbs.

That's the polymorphic solution. The system doesn't even need to know anything has changed.

---

It's really nice if you can pull that off. By encoding a meaningful rejection of an assumption into a value you don't have to force the question to change.

How many apples will you owe me if I give you 1 apple? -1. Because I owed you 2 apples from yesterday. Here the assumption of the question, "you will owe me", is rejected with a negative number. Even though the question was wrong, meaningful info is encoded in this answer. By rejecting it in a meaningful way the question isn't forced to change.

However, sometimes changing the question actually is the way to go. If the assumption can be made more reliable, while still useful, consider doing that.

Your problem is that, normally, updates come from players. But you've discovered a need for an update that doesn't come from player 1 or player 2. You need an update that says time has expired. Neither player is saying this so what should you do? Leaving player null seems so tempting. But it destroys information. You're trying to encode knowledge in a black hole.

The player field doesn't tell you who just moved. You think it does but this problem proves that's a lie. The player field tells you where the update came from. Your time expired update is coming from the game clock. My recommendation is give the clock the credit it deserves and change the name of the player field to something like source. Then the value can be player 1, player 2, or clock.

This way if the server is shutting down and has to suspend the game it can send out an update that reports what's happening and lists itself as the source of the update.

Does this mean that you should never just leave something out? No. But you need to consider how well nothing can be assumed to really mean a single well known good default value. Nothing is really easy to over use. So be careful when you use it. There is a school of thought that embraces favoring nothing. It's called convention over configuration.

I like that myself but only when the convention is clearly communicated in some way. It should not be a way to haze the newbies. But even if you're using that, null is still not the best way to do it. Null is a dangerous nothing. Null pretends to be something you can use right up until you use it, then it blows a hole in the universe (breaks your semantic model). Unless blowing a hole in the universe is the behavior you need why are you messing with this? Use something else.

Answer 2

null is not really the problem here. The problem is how most current programming languages deal with missing information; they do not take this problem seriously (or at least dont provide the support and safety most earthlings need to avoid the pitfalls). This leads to all the problems we've learned to fear (Javas NullPointerException, Segfaults, ...).

Lets see how to deal with that issue in a better way.

Others suggested the Null-Object Pattern. While i think it is applicable sometimes, there are a lot of scenarios where there is just no one-size-fits-all default value. In those cases, consumers of the possibly-absent information have to be able to decide for themselves what to do if that information is missing.

There are programming languages that provide safety against null pointers (so called void safety) at compile time. To give some modern examples: Kotlin, Rust, Swift. In those languages, the problem of missing information has been taken seriously and using null there is totally painless - the compiler will stop you from dereferencing nullpointers.
In Java, efforts have been made to establish the @Nullable/@NotNull annotations together with compiler+IDE plugins to reach the same effect. It was not really successful but thats out of scope for this answer.

An explicit, generic type that denotes possible absence is another way to deal with it. E.g. in Java there is java.util.Optional. It can work:
On a project or company level, you can establish rules/guidelines

• use only high-quality 3rd party libraries
• always use java.util.Optional instead of null

Your languages community / ecosystem may have come up with other ways of adressing this problem better than the compiler/interpreter does.

Answer 3

I do not see any merit in the statement that null is bad. I checked the discussions about it and they only make me impatient and irritated.

Null can be used wrong, as a "magic value", when it actually means something. But you would have to do some pretty twisted programming to get there.

Null should mean "not there", that is not created (yet) or (already) gone or not provided if it is an argument. It is not a state, the whole thing is missing. In an OO setting where things are created and destroyed all the time that is a valid, meaningful condition different from any state.

With null you get smacked at runtime where you get smacked at compile time with some sort of type-safe null alternative.

Not entirely true, I can get a warning for not checking for null before using the variable. And it is not the same. I may not want to spare the resources of an object that has no purpose. And more importantly, I will have to check for the alternative just the same in order to get desired behavior. If I forget to do that I would rather get a NullReferenceException at runtime than some undefined/unintended behavior.

There is one issue to be aware of. In a multi-threaded context you would have to safeguard against race conditions by assigning to a local variable first before performing the check for null.

The problem with null is it can mean far to many things. So it just ends up destroying information.

No, it does/should not mean anything so there is nothing to destroy. The name and type of the variable/argument will always tell what is missing, that is all there is to know.

Edit:

I am halfway the video candied_orange linked to in one of his other answers about functional programming and it is very interesting. I can see the usefulness of it in certain scenarios. The functional approach that I have seen so far, with pieces of code flying around, typically makes me cringe when used in an OO setting. But I guess it has its place. Somewhere. And I guess there will be languages that do a good job hiding what I perceive to be a confusing mess whenever I encounter it in C#, languages that provide a clean and workable model instead.

If that functional model is your mind set/framework, there is really no alternative to pushing any mishaps forward as documented error descriptions and ultimately let the caller deal with the accumulated garbage that was dragged along all the way back to the point of initiation. Apparently this is just how functional programming works. Call it a limitation, call it a new paradigm. You may regard it to be a hack for the functional disciples that enables them to continue on the unholy path a bit further, you may be delighted by this new way of programming that unleashes exciting new possibilities. Whatever, it seems pointless to me to step into that world, point back to the traditional OO way of doing things (yes, we can throw exceptions, sorry), pick some concept from it, pull it out of context and scream blue murder because it does not fit your recently discovered new world.

Any concept can be used the wrong way. From where I stand, the presented "solutions" are no alternatives for an appropriate use of null. They are concepts from another world.

Answer 4

Your question.

But, but - from my naivety, let me scream - sometimes a value CAN be meaningfully absent!

Fully on board with you there. However, there are some caveats I'll get into in a second. But first:

Nulls are evil and should be avoided whenever possible.

I think this is a well-intentioned but overgeneralized statement.

Nulls are not evil. They are not an antipattern. They are not something that is to be avoided at all costs. However, nulls are prone to error (from failing to check for null).

But I don't understand how failing to check for null is somehow proof that null is inherently wrong to use.

If null is evil, then so are array indexes and C++ pointers. They're not. They're easy to shoot yourself in the foot with, but they are not supposed to be avoided at all costs.

For the rest of this answer, I am going to adapt the intention of "nulls are evil" to a more nuanced "nulls should be handled responsibly".

---

Your solution.

While I agree with your opinion on the use of null as a global rule; I don't agree with your use of null in this particular case.

To summarize the below feedback: you're misunderstanding the purpose of inheritance and polymorphism. While your argument to use null has validity, your further "proof" of why the other way is bad is built on misuse of inheritance, rendering your points somewhat irrelevant to the null problem.

Most updates naturally have a Player associated with them - after all, it is necessary to know which player made which move this turn. However, some updates can't have a Player meaningfully associated with them.

If these updates are meaningfully different (which they are), then you need two separate update types.

Consider messages, for example. You have error messages and IM chat messages. But while they may contain very similar data (a string message and a sender), they do not behave the same way. They should be used separately, as different classes.

What you're doing now is effectively differentiating between two types of update based on the null-ness of the Player property. That is equivalent to deciding between an IM message and an error message based on the existence of an error code. It works, but it's not the best approach.

• The JS code will now simply receive an object without Player as a defined property, which is the same as if it was null since both cases require checking if the value is present or absent;

Your argument relies on misues of polymorphism. If you have a method which returns a GameUpdate object, it generally shouldn't ever care to differentiate between GameUpdate, PlayerGameUpdate or NewlyDevelopedGameUpdate.

A base class needs to have a fully working contract, i.e. its properties are defined on the base class and not on the derived class (that being said, the value of these base properties can of course be overridden in the derived classes).

This renders your argument moot. In good practice, you should never care that your GameUpdate object does or doesn't have a player. If you're trying to access the Player property of a GameUpdate, then you're doing something illogical.

Typed languages such as C# wouldn't even allow you to try and access the Player property of a GameUpdate because GameUpdate simply doesn't have the property. Javascript is considerably more lax in its approach (and it doesn't require precompilation) so it doesn't bother to correct you and has it blow up at runtime instead.

• Should another nullable field be added to the GameUpdate class, I would have to be able to use multiple inheritance to continue with this desing - but MI is evil in its own right (according to more experienced programmers) and more importantly, C# doesn't have it so I can't use it.

You shouldn't be creating classes based on the addition of nullable properties. You should be creating classes based on functionally unique types of game updates.

---

How to avoid the issues with null in general?

I do think it's relevant to elaborate on how you can meaningfully express the absence of a value. This is a collection of solutions (or bad approaches) in no particular order.

1. Use null anyway.

This is the easiest approach. However, you are signing yourself up for a ton of null checks and (when failing to do so) troubleshooting null reference exceptions.

2. Use a non-null meaningless value

A simple example here is indexOf():

"abcde".indexOf("b"); // 1
"abcde".indexOf("f"); // -1

Instead of null, you get -1. On a technical level, this is a valid integer value. However, a negative value is a nonsensical answer as indexes are expected to be positive integers.

Logically, this can only be used for cases where the return type allows for more possible values than can be meaningfully returned (indexOf = positive integers; int = negative and positive integers)

For reference types, you can still apply this principle. For example, if you have an entity with an integer ID, you could return a dummy object with its ID set to -1, as opposed to null.
You simply have to define a "dummy state" by which you can measure if an object is actually usable or not.

Note that you shouldn't rely on predetermined string values if you do not control the string value. You might consider setting a user's name to "null" when you want to return an empty object, but you'd encounter issues when Christopher Null signs up for your service.

3. Throw an exception instead.

No. Just no. Exceptions should not be handled for logical flow.

That being said, there are some cases where you don't want to hide the (nullreference) exception, but rather show an appropriate exception. For example:

var existingPerson = personRepository.GetById(123);

This can throw a PersonNotFoundException (or similar) when there is no person with ID 123.

Somewhat obviously, this is only acceptable for cases where not finding an item makes it impossible to do any further processing. If not finding an item is possible and the application can continue, then you should NEVER throw an exception. I cannot stress this enough.

Answer 5

The point of why nulls are bad is not that missing value is encoded as null, but that any reference value can be null. If you have C#, you are probably lost anyway. I don't see a point ro use some Optional there because a reference type is already optional. But for Java there is @Notnull annotations, which you seem to be able to set up at package level, then for values which can be missed you could use annotation @Nullable.

Answer 6

C.A.R. Hoare called nulls his "billion dollar mistake". However, it's important to note that he thought the solution wasn't "no nulls anywhere" but instead "non-nullable pointers as the default and nulls only where they're semantically required".

The problem with null in the languages that repeat his mistake is that you can't say that a function expects non-nullable data; it's fundamentally inexpressible in the language. That forces functions to include a lot of useless extraneous null-handling boilerplate, and forgetting a null check is a common source of bugs.

To hack around that fundamental lack of expressiveness, some communities (e.g. the Scala community) don't use null. In Scala, if you want a nullable value of type T, you take an argument of type Option[T], and if you want a non-nullable value you just take a T. If someone passes a null into your code, your code will blow up. However, it's considered that the calling code is the buggy code. Option is a pretty nice replacement for null, though, because common null-handling patterns are extracted into library functions like .map(f) or .getOrElse(someDefault).

Answer 7

Nulls are perfectly fine in JavaScript. They are evil in Java. Whether nulls are evil or not is language-specific, since it depends on how nulls are handled by the type system.

You are absolutely correct that sometimes a value can be meaningfully absent, and you need some way to indicate that. The problem with using nulls in Java is that all object references can be null, so the type system does not indicate if a particular value is optional or not. You have to somehow know this, and remember to check in all cases where the value can be absent. Since programmers otherwise rely on the type system to ensure an operation is legal, this is a frequent source of bugs.

In comparison TypeScript does it right. Nulls are explicit in the type signatures, so you always know if a value could be null or not. If you call a method on a nullable value without checking for null first, you get a compiler warning.

The fundamental difference is that in TypeScript null is its own type, while in Java null is a member of all object types.

JavaScript in comparison is dynamically typed, so the type system does not make any static guarantees of any types. So null is not worse or better than any other type.

C# had the same problem as Java until version 8, when it introduced explicit nullability in the type system. (As an optional feature though, you have to enable it.)

Allow me a rant: The "nulls are evil" slogan is a symptom of a big problem in the industry: People see some advice pertaining to a particular language or context, but then ignores the context and use it as a general rule. The famous "Billion Dollar Mistake" talk by C.E Hoare makes it clear it is not null's per se that is the problem, the problem is making all object reference types nullable by default.

If you try to avoid nulls in language like JavaScript or TypeScript, you will likely come up with something worse, since null's does not have this problem in the first place.

If you use Java you should use something like Optional<T> instead of nulls.

Answer 8

Nulls are not evil, there is realistically no way to implement any of the alternatives without at some point dealing with something that looks like a null.

Nulls are however dangerous. Just like any other low level construct if you get it wrong there's nothing below to catch you.

I think there are a lot of valid arguments against null:

• That if you are already in a high level domain, there are safer patterns than using the low-level model.

• Unless you need the advantages of a low-level system and are prepared to be suitable cautious then maybe you shouldn't be using a low-level language.

• etc...

These are all valid and further not having to make the effort of writing getters and setters etc is seen as a common reason not to have followed that advice. It's not really surprising a lot of programmers have come to the conclusion that nulls are dangerous and lazy (a bad combination!), but like so many other bits of "universal" advice they are not as universal as they are worded.

The good people who wrote the language thought they would be useful for someone. If you understand the objections and still think they are right for you, none else will know any better.

Answer 9

If someone tells you “X is evil” then you know they are anywhere between exaggerating, clueless or lying. Same with null pointers.

Take a simplified example where you would want to record a person and their spouse. So a “Person” object has a pointer to the spouse which is also a “Person” object. And Mr. X has no spouse.

There’s a horrible solution to this and a simple solution. The horrible one is to have a special instance of “Person” which is used to refer to non-existing persons. Now first you need a new property or method telling you that a Person instance is that nonexisting person. And all other properties must return default values. What’s the gender of a non-existing person? It can’t be male, or female, or transgender etc. Because the person doesn’t exist. So the gender should be “non-existing”. Now you need to add a case to every switch statement. All in all, a total mess. Or you use the good solution: Store a null pointer.

Null pointers are not the problem. Pointers where the compiler cannot know whether they can be null or not, that’s the problem. In a modern language you have non-nullable pointers: They can’t be null. You can’t even check whether they can be null because they can’t. And you have nullable pointers: They can be null, and you can’t access the object without verifying that the pointer is not null.

So if you are the wedding registrar and Mr.X wants to marry Mr.Y or Mrs. Z then you check the nullable spouse pointers, and if one is not null you arrest them for bigamy.

Answer 10

I think the idea that null is a big mistake is a case of looking at the problem backwards.

Yes, a failure to check for null is very prone to making your code go boom--but I consider that a feature. The alternatives always come down to providing an answer that is sometimes wrong--and in most cases wrong code that runs is worse than wrong code that goes boom. (The exception would be in situations where failure isn't an option--and those should be handled with exceptions to try to cut out the failing section and hope the rest can continue. I'm thinking of the first launch of the Ariane V rocket--an untrapped division underflow crashed a program whose usefulness had ended at "ignition sequence start" and lead to the loss of the rocket.)

Answer 11

I don't understand why anyone would say that "nulls are evil." IS NULL is a fundamental concept in every database. nil or NULL or isnull are fundamental to every programming language. But it's worth mentioning that the two concepts are not at all the same, and some unwanted ambiguity can arise there.

Answer 12

Java has an Optional class with an explicit ifPresent + lambda to access any value safely. This can be made in JS too:

See this StackOverflow question.

By the way: many a purist will find this usage dubious, but I do not think so. Optional features do exist.

Answer 13

I'd like to elaborate on my previous comment.

• In the "C" programming language, the word NULL is used where other languages use the term nil or nullptr. This is the value that a pointer takes when "it doesn't point to anything," and its numeric value is customarily zero.

• In SQL, however, NULL means "the absence of a value." However, this is a legitimate "setting" for the field to have. It is an explicit assertion that no value exists for this column in this record.

Therefore, the word NULL is ambiguous. It is not "evil" in any case – in fact, it is necessary. A linked-list must end somewhere. A tree-node might not currently point to as many things as it is capable of pointing to. But your program logic should never "stumble into" either a null-pointer nor a stale pointer value. And, if your program is going to "throw an exception," that exception should be meaningful and descriptive. It should describe what happened and, if it indicates the presence of a bug (not all "exceptions" do), should help you find it.

Answer 14

There are some very good answers already, but none that look closely at database nulls. If you declare a field as Integer default 0 then the database may return wrong results if zero actually means "not known". For example in Oracle, the SUM function will work fine, but the AVG function will return rubbish. If you had used NULL to mean "not known" then all is good - Oracle will ignore null values. I suspect other databases will act similarly.

As developers, we might claim we never use database SQL functions. That may be correct but what about a data analyst using a general purpose SQL reporting tool?

As a Java developer I follow the database example and use NULL where it makes good sense.