(On 2013.09.23, Aleksander Sumowski and I gave a talk on monads at the Developer Meetup, ThoughtWorks Pune. This post is a transcript of that talk.)

I have come across many people who are somewhat familiar with functional programming but have a hard time understanding the idea of monads. I believe monads are a simple concept, and the intent of this talk is to provide some intuition for them. And we promise not to use burritos in our explanation.

If you have not heard of monads before, this should still be approachable.

I will try to motivate the case for monads with some day-to-day code examples. The language used for the examples is Scala.

Example 1: Presence or absence of a value

You are given a string. You have to look up this string in a map named foo. The length of the value obtained is to be used as a key in another map, named bar, to get the final value.

This is how you might do it:

// snippet 1.1
import java.{util => ju}
import scala.collection.JavaConverters._
val fooJ: ju.Map[String, String] = Map("a" -> "xoxox", "b" -> "xoxo").asJava


val barJ: ju.Map[Int, String] = Map(4 -> "P", 5 -> "Q").asJava


def calc(s: String): String = {
  val x = fooJ.get(s)
  val y = barJ.get(x.length)
  y
}

But wait. This code is incorrect. What if the key is missing in the first map itself? In that case the map will return null, leading to a NullPointerException in the next operation.

Clearly we need something that guards us against null, so we add an if check.

// snippet 1.2
def calc(s: String): String = {
  val x = fooJ.get(s)
  val y = if (x != null) {
    barJ.get(x.length)
  } else {
    null
  }
  y
}

What was wrong with snippet 1.1 was that the two operations were sequenced wrongly. There were some additional effects necessary in between, and they were missing.

It would be nice if we could write code whose shape is like that of snippet 1.1 but which behaves like the code in snippet 1.2.

Example 2: Exceptions

You have to look up a couple of keys in a JSON object and then concatenate the obtained values with a space in between. Again, either or both keys might be absent, thus leading to an exception. In such an event, the exception should bubble up to the caller.

This is how you might do it:

// snippet 2.1
import play.api.libs.json._


def calc(json: JsObject): String = {
  val name = (json \ "name").as[String]
  val address = (json \ "address").as[String]
  name + " " + address
}

This code works as expected. The necessary sequencing, or effects, here are provided by a first-class language feature called exceptions. Imagine a language without exceptions.

What might you do in such a language? Perhaps send a tuple of error code and result, or some equivalent structure.

Is it possible to write code whose shape and behaviour are like that of snippet 2.1, but which does not use first-class exceptions?

Example 3: Futures and promises

A future is an abstraction of a value that will be available at some point in time. This is a concurrency abstraction invented in the late 1970s but popularised by Java and JavaScript. Scala’s variation is a bit better.

Consider a simple, unrealistic task. We need to make two web service calls. Something mandates that the second call must be made after the first one is over. Then, after obtaining both results, you sum the lengths of the response bodies.

In a good old serial, blocking, non-future API, this might look like this:

// snippet 3.1
def calc(s: String): Int = {
  val googleResponse = makeWebServiceCall("https://www.google.com/#q=" + s)
  val bingResponse = makeWebServiceCall("http://www.bing.com/search?q=" + s)
  googleResponse.body.length + bingResponse.body.length
}

Now let us rewrite this with an API that uses futures. How are we going to sequence operations in such an API? The popular solution seems to be callbacks. With such an approach the code might look like this:

// snippet 3.2
import concurrent.{Promise, Future}
import play.api.libs.ws.WS


def calc(s: String): Future[Int] = {
  val promise = Promise[Int]()
  val googleResponse = WS.url("https://www.google.com/#q=" + s).get()
  googleResponse onSuccess { case res1 =>
    val bingResponse = WS.url("http://www.bing.com/search?q=" + s).get()
    bingResponse onSuccess { case res2 =>
      promise.success(res1.body.length + res2.body.length)
    }
  }
  promise.future
}

The onSuccess here provides the necessary effects in between the operations.

This code is fairly straightforward, but it has the following problems:

  1. It exposes Promise, an implementation detail of futures.
  2. The onSuccess plus promise pattern gets repetitive fast and soon becomes an eyesore.

It would be nice if we could write code whose shape is like that of our serial code but which has future-like semantics.

Enter Type-Driven Development

Let me introduce you to a style of programming popular in statically typed functional languages: Type-Driven Development, or TyDD.

In TyDD, you encode as much semantic information as possible in your types and values. Of course there is a lot more to TyDD than that, but it is not the subject of this talk.

Let us revisit our examples and try applying TyDD to them.

Revisiting example 1

Scala has a data type named Option, which is a sum type that encodes presence or absence semantics. Languages with different type systems might use different encodings. It is roughly defined as:

// snippet 1.3
sealed abstract class Option[+A]
case class Some[+A](value: A) extends Option[A]
case object None extends Option[Nothing]

As you would expect, calling get on Scala’s Map returns an Option value. With Scala maps, our code might look like this:

// snippet 1.4
val fooS: Map[String, String] = Map("a" -> "xoxox", "b" -> "xoxo")


val barS: Map[Int, String] = Map(4 -> "P", 5 -> "Q")


def calc(s: String): Option[String] = {
  fooS.get(s) match {
    case Some(x) => barS.get(x.length)
    case None => None
  }
}

One advantage of the TyDD approach should be immediately clear by now: the compiler will not allow you to treat Option[A] as A, and so you cannot inadvertently write incorrect code like snippet 1.1. You are forced to deal with optionality explicitly.

But all that sequencing with pattern matching looks yucky, and it will only get yuckier with every additional operation. Wouldn’t it be nice if the data type in question could take care of that sequencing itself?

As it happens, Option does have a method that takes care of this sequencing. The method is called flatMap, and when we use it, the code looks like this:

// snippet 1.5
def calc(s: String): Option[String] = {
  fooS.get(s) flatMap { x =>
    barS.get(x.length) flatMap { y =>
      Some(y)
    }
  }
}

In the innermost block, you need to put the value back in Option. Some is the data constructor we use for that.

If you compare this code to snippet 1.1, you will notice that their shape is similar. The code is somewhat inverted along the vertical axis, but as you will soon see, there is a fix for that.

Revisiting example 2

Scala has a data type named Try that encodes success and failure semantics. It is roughly defined as:

// snippet 2.2
sealed abstract class Try[+A]
case class Success[+A](value: A) extends Try[A]
case class Failure(throwable: Throwable) extends Try[Nothing]

Let us imagine JsObject has a method named asTry[A] that returns Try[A]. With that, the code would look like this:

// snippet 2.3
import play.api.libs.json._
import util.{Try, Success, Failure}


def calc(json: JsObject): Try[String] = {
  (json \ "name").asTry[String] match {
    case Success(name) =>
      (json \ "address").asTry[String] match {
        case Success(address) => Success(name + " " + address)
        case Failure(ex) => Failure(ex)
      }
    case Failure(ex) => Failure(ex)
  }
}

That is a fair amount of boilerplate.

As you might expect at this point, Try also has a method named flatMap that helps you do away with this boilerplate. This is how the code looks with flatMap:

// snippet 2.4
import play.api.libs.json._
import util.{Try, Success, Failure}


def calc(json: JsObject): Try[String] = {
  (json \ "name").asTry[String] flatMap { name =>
    (json \ "address").asTry[String] flatMap { address =>
      Success(name + " " + address)
    }
  }
}

In the innermost block, you need to put the value in Try, and we use the Success data constructor for that.

This code is very similar in shape to the code in snippet 2.1.

Revisiting example 3

Our results are already wrapped in Future, so we are already using a distinct type to encode the semantics.

Now Future also happens to have a flatMap method. Let us rewrite the code in snippet 3.2 using it.

// snippet 3.3
import concurrent.{Promise, Future}
import play.api.libs.ws.WS


def calc(s: String): Future[Int] = {
  WS.url("https://www.google.com/#q=" + s).get() flatMap { googleResponse =>
    WS.url("http://www.bing.com/search?q=" + s).get() flatMap { bingResponse =>
      Future.successful(googleResponse.body.length + bingResponse.body.length)
    }
  }
}

In the innermost block, we create a future that is already completed with our value.

Again, it is similar in shape to the code in snippet 3.1.

Monad comprehensions

As we have seen, flatMap allows our code to have a simple shape, with the details of sequencing abstracted away. Now monads are so common in functional programming that functional languages often provide a syntactic sugar on top of them which makes the inversion along the vertical axis go away. This sugar is called monad comprehensions. Scala calls them for-comprehensions. The similarity to for loops is superficial and should be ignored.

Here is how our code snippets might look once we start using this notation:

// snippet 1.6
def calc(s: String): Option[String] = {
  for {
    x <- fooS.get(s)
    y <- barS.get(x.length)
  } yield y
}


// snippet 2.5
import play.api.libs.json._
import util.{Try, Success, Failure}


def calc(json: JsObject): Try[String] = {
  for {
    name <- (json \ "name").asTry[String]
    address <- (json \ "address").asTry[String]
  } yield name + " " + address
}


// snippet 3.4
import concurrent.{Promise, Future}
import play.api.libs.ws.WS


def calc(s: String): Future[Int] = {
  for {
    googleResponse <- WS.url("https://www.google.com/#q=" + s).get()
    bingResponse <- WS.url("http://www.bing.com/search?q=" + s).get()
  } yield googleResponse.body.length + bingResponse.body.length
}

These are even closer in appearance to snippets 1.1, 2.1, and 3.1 respectively.

Note that this is just syntactic sugar and compiles down, roughly, to the code we wrote before with flatMap.

So what is a monad?

A monad is essentially a two-method interface, which can be defined as follows:

// snippet 4.1
trait Monad[M[_]] {
  def flatMap[A, B](x: M[A], f: A => M[B]): M[B]
  def point[A](x: A): M[A]
  def map[A, B](x: M[A], f: A => B): M[B] = flatMap(x, a => point(f(a)))
}

I am using the word interface here in its generic sense, not in the OO sense. A more specific term for this abstraction mechanism would be type class, and you can learn more about it here.

About the methods:

  • flatMap: we already talked about it.
  • point: it is basically a method that allows you to put a value in the monad’s context in the innermost block.
  • map: it is defined by default in terms of flatMap and point, and you can override it if a more performant implementation specific to the data type is possible.

As an example, here is a monad implementation for Option:

// snippet 4.2
implicit object OptionMonad extends Monad[Option] {
  def flatMap[A, B](x: Option[A], f: A => Option[B]): Option[B] = x match {
    case Some(a) => f(a)
    case None => None
  }


  def point[A](x: A): Option[A] = Some(x)


  override def map[A, B](x: Option[A], f: A => B): Option[B] = x match {
    case Some(a) => Some(f(a))
    case None => None
  }
}

The implementation of a Monad type class also needs to obey some laws, which we will not go into here.

Intuition for monads

You use the monad abstraction when:

  1. You need customised sequencing for operations, that is, you need additional effects in between computations and want them abstracted away.
  2. You need to abstract over the sequencing details, that is, write code that is parameterised over M, some monad, and plug in a specific monad instance as necessary. This technique is used in Precog code to good effect.

Someone once described monads as something that lets you overload the semicolon. Since Scala does not use semicolons at the end of statements or expressions, the analogy is perhaps a little less vivid here, but the basic idea still holds.

Other monads

We have covered only three monads here, Option, Try, and Future, but there are many more:

  1. Seq for non-determinism.
  2. Either for binary type disjunction, often used for error handling.
  3. Reader for an implicit context passed around from which values can be read.
  4. Writer for logging.
  5. State for stateful computations.
  6. ST for localised mutation.
  7. Undo for the ability to undo and redo operations.

Kleisli composition

Given two functions f: A => B and g: B => C, it is fairly trivial to compose them together. Here is one way to write such a function:

// snippet 5.1
def compose[A, B, C](f: A => B, g: B => C): A => C =
  a => g(f(a))

Now what happens when I have two effectful functions instead, say f: A => M[B] and g: B => M[C]? Composing these is not trivial, because the output type of f and the input type of g do not match. We need a new composition function with a signature like this:

// snippet 5.2
def mcompose[A, B, C, M[_]](f: A => M[B], g: B => M[C]): A => M[C] =
  ???

It is not possible to write this generically without knowing something about M. How the operations are composed depends on the structure of M, so we let M tell us how to wire them up:

// snippet 5.3
def mcompose[A, B, C, M[_]](f: A => M[B], g: B => M[C])(implicit e: Monad[M]): A => M[C] =
  a => {
    val mb = f(a)
    e.flatMap(mb, { b =>
      val mc = g(b)
      e.flatMap(mc, { c =>
        e.point(c)
      })
    })
  }

This kind of composition is called Kleisli composition and is often denoted with the symbol >=>.

Monads are not alone

I hope this talk helped you gain some intuition for what a monad is and when to use this abstraction.

As it happens, monads are not alone. There is a whole family of type classes that let you abstract over computational patterns. Some key examples include Functor, Applicative, MonadPlus, Arrow, and Alternative. If you find monads interesting or useful, I recommend exploring the rest of the family as well. The best medium for doing that is probably Haskell, and here is a good book for learning the language.

Some more notes

  • The names flatMap and point are not standard, and different languages use different names. Some other names for flatMap are >>=, bind, m-bind, and SelectMany. Some other names for point are pure and return.
  • Scala’s Try violates certain monad laws and is therefore not quite a monad. However, those details are irrelevant to the basic nature of this talk. I could have used Either, but its implementation would require touching on too many tangential ideas.
  • A question often asked is whether monads are possible or useful in dynamic languages, given how centred they are around types. Yes, they are possible in dynamic languages, but much of their utility is lost, and they require a somewhat different encoding. I intend to blog about this separately.
  • Scala’s standard library does not have a Monad type class. How do its for-comprehensions work then? The answer is that the Scala compiler blindly translates them to flatMap and map method calls on that data type. Yes, map, not point. This approach has its pros and cons. Nevertheless, a Monad type class can be useful in many cases, and there are libraries that provide it.
  • If you want to understand monads more thoroughly, here is a very good tutorial. It is fairly long, but worth the time if the topic clicks for you.
  • Monad comprehension sugar is neat, but it still leaves much to be desired. People have taken the idea further in many different directions and come up with better syntaxes, such as this one.