More data, more problems: Going toe-to-toe with time series data using Postgres

Nils Norman Haukås

Lightning talk - in English

In this talk I would like to share a story of how I inherited a Node.js application crushed under the weight of time series data. This was millions of data points which had to be aggregated in order to yield insights and we struggled to do so.

The original application was built around a message-driven architecture with high hopes mixed in. After dealing with memory leaks in production for several months thereafter those hopes were squished. I'll give a postmortem of that architecture and how it failed us. There were some tough lessons there that we want to share.

So, do we give up or will we try to rearchitect the application?

After some tough discussions we got management buy-in to get one shot at rebuilding core parts of the application. We chose to leverage Postgres and thankfully it proved to be a good bet. I'll share the database model we landed on and the reasoning behind it.

Finally, I'll share what I learned on how to work effectively with Postgres.