New TinyBits Release

I have just released version 0.8.0 of TinyBits which brings bug fixes, adds features and improves performance! Let’s have a quick look at these

What’s New

• [BUG] Fix a lingering off by 1 error in blob encoding
• [FEATURE] Support keys as symbols in Ruby
• [PERF] Better hash distribution for faster dedupe strings search
• [PERF] Native dictionary support

The last point is big! since we introduced dictionary support it resulted in much better sizes for smaller documents but since it was done completely in Ruby, it made the performance suffer. Now we have the best of both worlds!

Quick Benchmarks

We used the same set of 11 large JSON documents we used in previous benchmarks, for testing smaller ones we extracted a record from each into its own document. For the dictionaries, we used a pretty standard dictionary for each document based only on its keys.

Gems Compared

• JSON 3.1.0
• MsgPack 1.8.6
• TinyBits 0.8.0

Document Size

To the left we see the averages of the documents from the large set. Here TinyBits does very well and slashes more than half of the original document size. Dictionaries take the documents size down a bit more, but the difference is not huge.

To the right we see the smallish documents, ~300 bytes in size. In these small sizes with hardly any key repetitions TinyBits is able to only achieve a 20% size reduction, mainly due to the packed binary format and the floating number compression. But once we add a dictionary, the weight of the keys disappears and the documents are now also less than half the original.

Encoding Rate

To the left we have the full documents being encoded and we can see that TinyBits in its two flavors is quite a bit faster than JSON and MsgPack. It’s great news that the space savings from using the dictionary did not affect the encoding speed, it’s even slightly faster on average!

To the right we look at the small records and here the different is more profound. JSON switches places with MsgPack here and gets ahead of it, but TinyBits is almost 1.5x the performance of JSON here.

Decoding Rate

With larger document decoding JSON is quite faster than MsgPack, TinyBits maintains a sizable advantage though with up to 33% faster decoding when using a dictionary.

For smaller records MsgPack claims a lot of the performance deficit and gets with 5% of JSON. TinyBits though is in a different league and with a dictionary is over 80% faster than JSON.

Conclusion

TinyBits is currently the fastest, most space and memory efficient binary encoder/decoder for Ruby apps. If you don’t require JSON’s readability or inter-operability then you cannot get better results at the moment!.