Easing the use of the Embedded Document pattern

Martin Fowler recently blogged about the Embedded Document pattern. It is a very convenient way of dealing with JSON documents in web apps, and we are using it for our models in the app we are currently working on.

In this post, we will see how we can abstract away the smaller patterns that recur when using the Embedded Document pattern. Here is the code snippet we will be working with, based on code from Fowler’s post:

1
class Delivery
2
  def initialize(data)
3
    @data = data
4
  end
5

6
  def customer
7
    @data['customer']
8
  end
9

10
  def ship_date
11
    Date.parse(@data['shipDate'])
12
  end
13
end
14

15
class Order
16
  def initialize(data)
17
    @data = data
18
  end
19

20
  def deliveries
21
    @data['deliveries'].map { |d| Delivery.new(d) }
22
  end
23

24
  def quantity_for(a_product)
25
    item = @data['items'].detect { |i| a_product == i['product'] }
26
    item ? item['quantity'] : 0
27
  end
28
end
29

30
order_hash = JSON.parse(
31
  '{
32
      "id": 1234,
33
      "customer": "martin",
34
      "items": [
35
        {"product": "talisker", "quantity": 500},
36
        {"product": "macallan", "quantity": 800},
37
        {"product": "ledaig",   "quantity": 1100}
38
      ],
39
      "deliveries": [
40
        { "id": 7722,
41
          "shipDate": "2013-04-19",
42
          "items": [
43
            {"product": "talisker", "quantity": 300},
44
            {"product": "ledaig",   "quantity": 500}
45
          ]
46
        },
47
        { "id": 6533,
48
          "shipDate": "2013-04-18",
49
          "items": [
50
            {"product": "talisker", "quantity": 200},
51
            {"product": "ledaig",   "quantity": 300},
52
            {"product": "macallan", "quantity": 300}
53
          ]
54
        }
55
      ]
56
   }'
57
)
58

59
order = Order.new(order_hash)
60
p order.deliveries
61
p order.quantity_for('talisker')

We can observe the following recurring patterns in the above code:

All of these classes need to have a constructor that takes a data object and stores it in a field.
Many attributes require only a simple lookup in the wrapped hash, for example Delivery#customer.
Some attributes need a lookup in the hash followed by a transformation, for example Delivery#ship_date.
In some cases, the nested hashes further need to be wrapped in objects. These nested objects can appear in sequences too, for example Order#deliveries.
Some attributes and methods involve complex operations and are too specific to derive any generalisation from them, for example Order#quantity_for.

Let us start by defining a class called EmbeddedDocument, which provides the base for such classes.

1
class EmbeddedDocument < Struct.new(:data)
2
end

This provides the necessary constructor to a subclass of EmbeddedDocument, thus taking care of item 1.

Before we go further, let us conjure up an abstraction named “embedder”. An embedder is nothing but an object with a method named call that takes one of the possible types in a JSON document, such as a number, a string, an array, a hash, or nil, and returns a more useful representation thereof, potentially wrapping it in some subclass of EmbeddedDocument.

Now we can define an identity embedder, something that returns a value as-is, without any transformations, which we will use for scalar values such as numbers, strings, and nil.

1
def scalar
2
  lambda { |x| x }
3
end

This takes care of item 2.

We can define some common transformations in a similar way and take care of item 3. We will define one for dates to serve as an example.

1
def date
2
  lambda { |x| Date.parse(x) }
3
end

We will deal with item 4 in two steps.

Step 1: We will make the class object associated with every EmbeddedDocument an embedder too. That is, it will have a call method that takes parsed data and returns an instance wrapping it.

1
class EmbeddedDocument < Struct.new(:data)
2
  def self.call(value)
3
    new(value)
4
  end
5
end

Step 2: We will now define a couple of functions that take one embedder and return another one. This is a very powerful technique from functional programming called combinatory design. What we are defining below can be referred to as embedder combinators.

1
def sequence_of(e)
2
  lambda { |array|
3
    array.map { |x| e.call(x) }
4
  }
5
end
6

7
def defaulted(e, default)
8
  lambda { |value|
9
    value.nil? ? default : e.call(value)
10
  }
11
end

Let us define a method key that will allow us to specify a key name and the embedder to be used.

1
class EmbeddedDocument < Struct.new(:data)
2
  def key(name, embedder)
3
    embedder.call(data[name])
4
  end
5

6
  def self.call(value)
7
    new(value)
8
  end
9
end

We now have all the machinery ready. This is how our original classes look with the new utility:

1
class Delivery < EmbeddedDocument
2
  def customer
3
    key('customer', scalar)
4
  end
5

6
  def ship_date
7
    key('shipDate', date)
8
  end
9
end
10

11
class Item < EmbeddedDocument
12
  def product
13
    key('product', scalar)
14
  end
15

16
  def quantity
17
    key('quantity', scalar)
18
  end
19
end
20

21
class Order < EmbeddedDocument
22
  def deliveries
23
    key('deliveries', sequence_of(Delivery))
24
  end
25

26
  def items
27
    key('items', sequence_of(Item))
28
  end
29

30
  def quantity_for(a_product)
31
    item = items.detect { |i| a_product == i.product }
32
    item ? item.quantity : 0
33
  end
34
end

Beautiful, isn’t it?

But this is Ruby, and we can do even better. Here are a couple of ways we can improve the solution above:

Define a method on EmbeddedDocument’s class that takes only the absolute essentials and defines a JSON accessor method for us.
We have many cases above in which a field access is a simple lookup in the wrapped hash and nothing more. Why not put method_missing to good use here?

Here is what our EmbeddedDocument class looks like after those improvements:

1
class EmbeddedDocument < Struct.new(:data)
2
  def self.key(name, embedder, accessor_name = nil)
3
    name = name.to_s
4
    accessor_name ||= name
5
    define_method(accessor_name) do
6
      embedder.call(data[name])
7
    end
8
  end
9

10
  def self.call(value)
11
    new(value)
12
  end
13

14
  def method_missing(sym)
15
    data[sym.to_s]
16
  end
17
end

Use:

1
class Item < EmbeddedDocument
2
end
3

4
class Delivery < EmbeddedDocument
5
  key :shipDate, date, :ship_date
6
  key :items, sequence_of(Item)
7
end
8

9
class Order < EmbeddedDocument
10
  key :deliveries, sequence_of(Delivery)
11
  key :items, sequence_of(Item)
12

13
  def quantity_for(a_product)
14
    item = items.detect { |i| a_product == i.product }
15
    item ? item.quantity : 0
16
  end
17
end

This is even more concise, and I think it is an improvement on the earlier version.