Scala Callback Pyramid of Doom

Question

I would like to solicit some general design principles and best practices to avoid creating a callback pyramid of doom particularly in the Scala language.

Consider the following rudimentary and imaginary code snippet:

val response: Future[Either[ApiError, Option[CogWidget]]] = services.findCogWidget("Some-UUID-42")
response.flatMap {
  case Right(cogWidget) =>
    val validationResponse: Future[Either[ApiError, ValidationResult]] = services.validateCogWidget(cogWidget)

    validationResponse.flatMap {
      case Right(validationResult) =>
        val registerResult: Future[Either[ApiError, RegistrationTicket]] = services.cogWidgetRegisterRequest(cogWidget, validationResult)
        registerResult.map {
          case Right(ticket) =>
            Some(":D" -> ticket)

          case _ => None
        }

      case x @ Left(_) => Future.successful(x)
    }      

  case x @ Left(_) => Future.successful(x)
}

How could you rewrite this to avoid / eliminate the callback pyramid of doom effect?

Answer 1

First of all, I encourage you to give it a try without the Eithers if possible. Futures already encode an error state, and that's sufficient in the vast majority of real world cases. That gets you down to a single monad, which lets you use for comprehensions like this (I made some mods to make the example self contained and compilable):

import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.Future

object PyramidOfDoomNoEither {
  case class Ticket()
  def findCogWidget(cog: String): Future[Option[Int]] =
    Future.successful(Some(1))
  def validateCogWidget(widget: Option[Int]): Future[Boolean] =
    Future.successful(true)
  def cogWidgetRegisterRequest(widget: Option[Int], valid: Boolean): Future[Option[Ticket]] =
    Future.successful(Some(Ticket()))

  def processWidget(cog: String): Future[Option[Ticket]] = {
    for {
      widget <- findCogWidget(cog)
      valid  <- validateCogWidget(widget)
      ticket <- cogWidgetRegisterRequest(widget, valid)
    } yield ticket
  }
}

Another recommendation is to think of pattern matching as a last resort. It's a very general purpose tool, which is why I think a lot of functional programmers take to it like a drug, but it also makes it a sort of blunt instrument. If you dig deeper, there's usually a more precise special purpose tool that can get the job done more elegantly.

If you insist on keeping the Either, one thing that will help is to use right projections (now the default in 2.12) and short-circuit mapping instead of pattern matching. However, you're not going to get the ideal brevity without digging into monad transformers. Scalaz has a nice set of ready-made ones, although their Either annoyingly goes by the \/ symbol.

Basically, the problem here is that for comprehensions can only <- across one monad. This is cool when you just have the Future like my example above, but you asked how to <- across both the Future and the Either monads at the same time. The solution is to use EitherT to squish the Future and Either into one big FutureEither monad, and everyone is happy. All that sounds kind of scary, but remember, we only want to introduce all this monad stuff if and when it simplifies our code, and in this case, I hope you agree it does:

import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.Future
import scalaz.EitherT
import scalaz.std.scalaFuture.futureInstance
import scalaz.{\/,-\/,\/-}

object PyramidOfDoom {
  case class Ticket()
  def findCogWidget(cog: String): Future[String \/ Option[Int]] =
    Future.successful(\/-(Some(1)))
  def validateCogWidget(widget: Option[Int]): Future[String \/ Boolean] =
    Future.successful(\/-(true))
  def cogWidgetRegisterRequest(widget: Option[Int], valid: Boolean): Future[String \/ Option[Ticket]] =
    Future.successful(\/-(Some(Ticket())))

  def processWidget(cog: String): Future[String \/ Option[Ticket]] = {
    val ticket = for {
      widget <- EitherT(findCogWidget(cog))
      valid  <- EitherT(validateCogWidget(widget))
      ticket <- EitherT(cogWidgetRegisterRequest(widget, valid))
    } yield ticket
    ticket.run
  }
}

Really, the hardest part here is knowing that EitherT exists, that it's applicable in simplifying this kind of situation, and being able to decipher the terse documentation. Easier said than done, but I've found that once I saw a few examples that showed me what was possible, I was able to iterate on my code until it got pretty close, even if it took quite a while at first.