Cloud Hosting

Image by midjourney

Modern applications almost exclusively store their state in databases and also read any state they require to perform their tasks from databases. Well concern ourselves with adding resilience to the processes of reading from and writing to these databases, making them highly reliable.

The obvious way to do this is to improve the quality of the hardware and software comprising the database so our reads and writes never fail. But this becomes a law of diminishing returns where once were already at high availabilities, pouring more money in moves the needle only marginally. Adding redundancy to achieve high availability quickly becomes a much better strategy.

So, what does this high reliability via adding redundancy to the architecture look like? We remove single points of failure by spending more money on redundant systems. For example, maintaining redundant copies of the data so that if one copy gets corrupted or damaged, the others can be used to repair. Another example is having a redundant database which can be read from and written to when the primary one is unavailable. Well call these kinds of solutions where additional memory, disk space, hardware or other physical resources are allotted to ensure high availability spatial redundancy. But can we get high reliability (going beyond the characteristics of the underlying databases and other components) without spending any additional money? Thats where the idea of temporal redundancy comes in.

All images in this article unless otherwise specified are by the author.

If spatial redundancy is running with redundant infrastructure, then temporal redundancy is running more with existing infrastructure.

Temporal redundancy is typically much cheaper than spatial redundancy. It can also be easier to implement.

The idea is that when reliability compromising events happen to our applications and databases, they tend to be restricted to certain windows in time. If the

Read more from the original source:

Adding Temporal Resiliency to Data Science Applications | by Rohit Pandey | Mar, 2024 - Towards Data Science