WAL-RUS: a Rust Rewrite of WAL-G for PostgreSQL Backups
ClickHouse Cloud, a managed Postgres service provider, encountered memory predictability issues with WAL-G, a mature and battle-tested Postgres backup and WAL archival tool written in Go. As they scaled Postgres into resource-constrained environments, they sought a backup system with a more predictable memory profile. WAL-G's garbage-collected runtime made memory usage harder to predict, with peak virtual memory reaching nearly 2.8 GB during benchmarking. In contrast, WAL-RUS, built using Rust, maintained a stable memory profile below 1 GB, reducing memory consumption by over 70%. This development matters because predictable memory usage is crucial for operators to confidently allocate resources to Postgres and ensure reliable performance.
The emergence of WAL-RUS reflects the growing importance of efficient resource utilization in cloud-native infrastructure. As cloud providers and managed service operators continue to optimize their offerings for cost and performance, tools like WAL-RUS will play a critical role in enabling more efficient use of resources. This development also highlights the trade-offs between mature, battle-tested tools like WAL-G and newer, more efficient alternatives like WAL-RUS. While WAL-G remains a reliable choice, WAL-RUS offers a more predictable memory profile, making it an attractive option for resource-constrained environments.
The implications of WAL-RUS are significant, as it provides a compatible alternative to WAL-G with improved memory efficiency. As ClickHouse Cloud plans to make WAL-RUS the default backup and WAL archival mechanism for their managed Postgres offering, it will be essential to monitor its performance and adoption. Additionally, the project's open-source nature and compatibility with WAL-G configuration and archives make it an attractive option for existing deployments. However, operators should carefully evaluate the trade-offs between WAL-G and WAL-RUS, considering factors like functionality, performance, and support.
Key Takeaways
ClickHouse Cloud developed WAL-RUS to address memory predictability issues with WAL-G, reducing peak virtual memory by over 70%.
WAL-RUS maintains compatibility with WAL-G configuration and archives, making adoption straightforward for existing deployments.
WAL-RUS uses Rust's explicit memory management and a daemonized streaming architecture to achieve archival throughput comparable to WAL-G.
The project is open source, and ClickHouse Cloud welcomes feedback, testing, and contributions.
About the Source
This analysis is based on reporting by Hacker News. Here is a short excerpt for context:
CommentsRead the original at Hacker News
