Scaling Akvorado BMP RIB with Sharding

Akvorado, a tool that associates routing information with flows, has overcome a major scaling hurdle by adopting sharding and lock-free reads. The initial implementation, which stored routes in a map and interned values to save memory, was bottlenecked by a global read/write lock. With the introduction of sharding, the RIB is split into multiple shards, each handling a subset of prefixes, allowing concurrent updates and significantly improving read and write latencies. The sharding approach also enables lock-free reads, further reducing contention and latency. The improvements were demonstrated through benchmarking, which showed that the number of writers can degrade read latency, highlighting the need for careful tuning.

The adoption of sharding and lock-free reads in Akvorado's RIB implementation reflects the broader trend of distributed systems and databases moving towards sharding and lock-free designs to improve scalability and performance. This shift is driven by the increasing complexity and size of data sets, as well as the need for real-time processing and low-latency operations. Other systems, such as databases and messaging platforms, are also exploring similar approaches to improve their performance and scalability.

The implications of Akvorado's sharding and lock-free reads implementation are significant, as it enables the tool to handle tens of millions of routes efficiently. However, the risk of degradation in read latency with high numbers of writers highlights the need for careful tuning and monitoring of the system. Additionally, the use of lock-free reads introduces the possibility of stale prefix indices, which requires careful management to avoid issues. As Akvorado continues to evolve, it will be interesting to see how the developers address these challenges and further optimize the performance of the RIB.