Trigger warning: this post has nothing to do with AI. It’s just “old” school software engineering that’ll only be interesting if your job involves building APIs and making systems scalable.

Phew, that out the way: Mike and the On-call team have been working to find a scaling strategy for our alerts and other APIs that have been a bit slow for our larger customers who have many millions of resources.

Building APIs that provide fast filtering and pagination is a tricky undertaking. I struggled a lot with this at GoCardless where listing payments with arbitrary filters was always a pain: new edge cases each week, and even the more solid optimisations would breakdown in the face of Postgres statistic bugs or query planner missteps.

Thankfully, we’ve found a solution that uses bloom filters to make filtering and pagination fast no matter how much data you have in the system. With the number of alerts we receive >10x’ing each year, I’m happy we’ve landed on a trick that should keep things snappy for the next few years at least.

The write-up (link in thread) explains the problem, what we tried before landing on bloom filters (gin indexes and more) and why this was the eventual winner.

A fun Sunday read that gives me nostalgia for the pre-AI days!