Not-so-embedded documents with Clojure

In my previous post, I talked about the Embedded Document pattern, and about how a combination of combinators and a little metaprogramming can abstract away its recurring bits. When I wrote that post, I wondered how I might deal with the same problem in Clojure, a language that lacks OO in the traditional sense. I fiddled with it a bit and ended up with what I find to be a reasonably elegant solution. This post talks about that.

In Clojure, we like to keep our data structures naked. We do not wrap them in classes unless there is a good reason to do so. This has the huge advantage that all the functions and utilities available for maps, sets, and sequences still work with your data.

Our initial approach to the problem might look like this:

1
(def data
2
  {:id "product1234"
3
   :name "Nex-5R"
4
   :href "totoro.com"
5
   :links [{:id "link1"
6
            :url "kokoro.com"}
7
           {:id "link2"
8
            :url "boboro.com"}]})
9

10
(defn product-master-id
11
  [product]
12
  (:name product))
13

14
(defn link-master-id
15
  [link]
16
  (:id link))
17

18
;; Collect product master ID and master IDs for all its links
19

20
{:product-master-id (product-master-id data)
21
 :links-master-ids (map link-master-id (:links data))}

Pretty simple. Just data and functions.

However, there is a disadvantage with this approach: the caller always has to know the type of the input data, whether it is a link, a product, and so on. In a system involving complex data comprising dozens of models, this would become very tedious. It is not ideal.

Wouldn’t it be nice if we could have our data respond to function calls polymorphically, without having to wrap it in some type or otherwise pollute the data? The good news is that there are a couple of Clojure features that make this possible:

Metadata: most Clojure data structures implement the clojure.lang.IObj protocol, which provides a way to decorate references with metadata without affecting the data itself. We can store “type tags” for our data in that metadata.
Multimethods: these let you dispatch on an arbitrary function. In our case, we could use a dispatch function that looks up the type tag embedded in the metadata.

With that, our design might look like this:

1
(defn with-type-tag
2
  [data type-tag]
3
  (with-meta data {:type-tag type-tag}))
4

5
(defn type-tag
6
  [data]
7
  (:type-tag (meta data)))
8

9
(defn tag-as
10
  [tag]
11
  (fn [data] (with-type-tag data tag)))
12

13
(defmulti master-id type-tag)
14

15
(defmethod master-id ::Product
16
  [this]
17
  (:name this))
18

19
(defmethod master-id ::Link
20
  [this]
21
  (:id this))
22

23
(defmulti links type-tag)
24

25
(defmethod links ::Product
26
  [this]
27
  (->> this :links (map (tag-as ::Link))))
28

29
;; Collect product master ID and master IDs for all its links
30

31
{:product-master-id (master-id data)
32
 :links-master-ids (map master-id (links data))}

Great, we got a custom tagging and dispatching system working with a tiny bit of code.

Since we are always dispatching on a type tag, we could spin up a small macro that avoids some of this duplication for us.

1
(defmacro defmessage
2
  [name' tag & forms]
3
  `(do
4
     (defmulti ~name' type-tag)
5
     (defmethod ~name' ~tag
6
       ~@forms)))
7

8
;; Here is how the relevant part of the previous snippet looks after we start
9
;; using this macro
10

11
(defmessage master-id ::Product
12
  [this]
13
  (:name this))
14

15
(defmessage master-id ::Link
16
  [this]
17
  (:id this))
18

19
(defmessage links ::Product
20
  [this]
21
  (->> this :links (map (tag-as ::Link))))

Cool. With just a few more lines, we also have a custom syntax for our new system.

What you see here is not a novel idea, but something used very routinely in Lisps. Some refer to it as a “custom type system”, though I personally am not a fan of that term, because “type” means something entirely different to me.

We have done a pretty fine job here, but there is a lot of room for improvement:

There could be messages, that is, tag-dispatched polymorphic functions, with arity greater than one. The dispatch function should handle that.
We could define a variation of defmessage that takes a function value. This would be useful for item 3.
We could define combinators for common cases, such as “sequence of type-tag”.
We could add some sugar that lets one define messages for a model together in one place. Note that this would look somewhat like OO classes syntactically, but it would not limit extensibility, because what we are essentially writing are multimethods.
You could extend this further to include things like validations. A macro for a model could generate a tagger that runs those validations before tagging the given data.

I will leave all of that for readers who feel like taking the idea a bit further.

If you found this post interesting, here are a few more links and resources that might strike your fancy: