Oj vs JSON vs MessagePack vs CBOR

Introduction

It had been a while since I had to look at serialization performance in Ruby. I generally would just use MessagePack for performance, and if I needed human readability I would opt for the Oj json gem. But due to my current project heavily relying on efficiently encoding, transmitting and then decoding data I thought it was time to revisit this topic again and find out which solution works best for my use case.

Requirements

I drafted a set of requirements to come up with metrics that could help identify which serialization option work best for my use case

Works well with a wide range of document types

My project is very data heavy and deals with a wide range of document types so I need the serializer to not have glaring weaknesses in some areas (e.g. it becomes very slow when storing fractions). This also means we favor schemaless formats as we can adapt quickly to these different structures. That means no protocol buffers for this test!.

Efficient encoding output size

Needless to say, I will be sending this data over the wire. And this needs to be minimized to make sure we don’t consume bandwidth unnecessarily. 

Efficient encoding and decoding times, decoding being more important

This is very critical, as I don’t want to degrade the performance of the apps I am sourcing the data from by being a CPU hog, everything needs to be fast and swift

Efficient memory usage

In the same vein of the previous point, we need to be lean resource usage wise. Triggering the GC often for the host app is as bad as running slow.

Portable, not tied to Ruby

Since data processing goes through multiple steps, with multiple tools involved, thus the data format has to be understandable by all these tools Thus Ruby’s Marshal is not an option.. 

Being human readable is totally not required 

It will be code sending data to code for processing and then sent to more code for storage and later fetched by code for presentation. Having human readability is completely not necessary. Doesn’t mean JSON is not a contender, just that it gets no bonus points for being human readable

The Contenders

Oj, the optimized JSON gem

For years, Oj has been the de facto gem to use when rendering or parsing JSON documents in Ruby. The implementation was, a lot of emphasis on was, a lot faster than the standard library JSON module

JSON, the standard library JSON gem

The standard library had the JSON gem since eons, but it was not particularly known for performance. One thing though is that this gem saw a lot of activity by Ruby core team members recently and it would be interesting to see how it shapes up at the moment.

MessagePack

The infamous message pack, lighting fast, with really compact output. The solution I would reach for whenever I needed to communicate data. But we will see if it can stand the test of time and stay ahead of the other contenders.

CBOR 

CBOR is another binary format, very similar to MessagePack but more on the formal side. I am expecting this to perform very similarly to MessagePack on all fronts but we will have to see how it fares.

Test Documents

I have asked some LLM to generate different documents with different characteristics and here are two sample records from these documents:

users.json

{
  "id": "usr_12345",
  "name": "Jane Smith",
  "email": "jane.smith@example.com",
  "role": "admin",
  "created_at": "2025-01-15T08:30:22Z",
  "preferences": {
    "notifications": true,
    "theme": "dark",
    "language": "en-US"
  },
  "address": {
    "street": "123 Main St",
    "city": "San Francisco",
    "state": "CA",
    "postal_code": "94105"
  }
}

geospatial.json

{
  "location_id": "loc_downtown_cafe",
  "name": "Downtown Coffee Shop",
  "coordinates": {
    "latitude": 37.7749,
    "longitude": -122.4194
  },
  "address": "123 Market St, San Francisco, CA",
  "type": "cafe",
  "polygon": [
    {"lat": 37.7748, "lng": -122.4193},
    {"lat": 37.7749, "lng": -122.4191},
    {"lat": 37.7751, "lng": -122.4192},
    {"lat": 37.7750, "lng": -122.4195},
    {"lat": 37.7748, "lng": -122.4193}
  ],
  "elevation": 12.5
}

events.json

{
  "event_id": "evt_abcdef123456",
  "event_type": "user_login",
  "timestamp": "2025-04-16T09:23:47Z",
  "actor": "usr_12345",
  "ip_address": "192.168.1.100",
  "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)",
  "metadata": {
    "login_method": "password",
    "device_id": "dev_iphone12_6789",
    "success": true
  }
}

And more, but you get the point.

Of each data type we generated 10 records in an array and fed that tot the different encoders/decoders.

Benchmarking Methodology

benchmark-ips and benchmark-memory were used to measure performance and memory usage.

All gems were used in their default configurations. For the MessagePack gem, we used the Packer/Unpacker classed for more efficient encoding/decoding than using the class methods pack/unpack

We also check the binary size and the effect of further compression both with LZ4 and Zstd (using their default settings)

All tests were run on the same machine (7840HS CPU) using Ruby 3.4 with YJIT enabled.

The benchmark code can be found in this gist.

Data compactness

In the following table, document size in bytes is reported for different serialization methods

Visually they look like this:

To the left you see a radar map showing each relative document size for each serialization method. To the right the same data is aggregated and is shown in a column chart.

As can be seen from the data, JSON produces the largest sized documents, with CBOR and MessagePack both besting it by ~15%.

Encoding Performance

Similar to data sizes, here’s a table showing encoding rate (as in docs encoded per second) across our 11 document types

The data is visualized as such

As can be seen, MessagePack leads the pack here in encoding performance, with CBOR coming second, JSON third and Oj finished in the last place. That’s quite the surprise as typically you would expect Oj to leave JSON in its dust but here we are. Kudos to the Ruby core team for this achievement!.

Still, MessagePack’s performance is unparalleled. It delivers 17%, 68% & almost 100% more encoding performance than CBOR, JOSN and Oj, respectively

Decoding Performance

The data for documents decoded per second was a bit surprising as can be seen below:

Not only is the JSON gem much faster than Oj in this benchmark, it is the fastest of all!, beating even the binary formats as well, sometimes by a big margin, in all but one document. What a turn of events!. The effort that went into optimizing that gem is really paying off here.

Encoding Memory Consumption

Memory consumption in MB during a run of 5K document encodings was measures as follows:

While encoding, MessagePack is the most memory efficient, followed closely by CBOR. Oj is comes third and is usually equal to or much better than the JSON gem which lands last.

Decoding Memory Consumption

Memory consumption in MB during a run of 5K buffer decodings was measured as follows:

As is tradition now, the JSON gem is the best in memory consumption during decoding. Followed closely by MsgPack, then by Oj. CBOR gets a deserved last place for consuming much more memory compared to the other solutions.

The JSON gem improvements

If you are using JSON in Ruby then do your self a favor and just use the default library JSON gem. That’s one more gem you don’t want to think of installing or adding to your Gemfile.

By looking at the C code, there seems to be a lot of effort that went into making the parsing phase truly fast. Things like using interned strings, and assigning elements to collections in bulk and caching seen before strings. All of these, and more work together to make the JSON gem run much much faster than before.

Kudos to the Ruby core team for this feat and for making our lives easier!.

Verdict

Given the mixed results it is hard to declare an absolute winner here. But maybe we can give some conditional recommendations:

1. If you are concerned with encoded size and encoding time over decoding time then go for MessagePack
2. If you are more concerned with decoding time then surely go for the JSON gem
3. If you are, like me, concerned with all three parameters then tough luck. Nothing serves all these needs at once at the moment. Maybe something else will appear soon? maybe? 😉

A note on space efficiency

We have seen how MessagePack and CBOR are more space efficient than the JSON format. It is worth noting though that most encoded data is usually compressed using compressors like Zlib, Zstd and LZ4. What holds true for the encoding phase might not hold true when compression is applied. And indeed, my initial testing shows that JSON documents are slightly more compressible than MessagePack and CBOR encoded ones.

I might follow up with some data on compressed sizes and maybe even encoding + compression and decompression + decoding times for each format.