.NET Architecture & System Design Interview Question #188
Why can autoscaling fail to solve an overloaded system?
Deployment, Cloud Operations, SRE & Senior System Design Scenarios Senior Advanced
Quick Interview Answer
Autoscaling one application tier does not create capacity in its dependencies. Adding web instances can increase pressure on a database, external API, connection pool, or shared cache and make the actual bottleneck worse.
Detailed Explanation
Autoscaling one application tier does not create capacity in its dependencies. Adding web instances can increase pressure on a database, external API, connection pool, or shared cache and make the actual bottleneck worse.
Architecture should identify the constrained resource and scale the system end to end. Autoscaling also reacts with delay, so sudden bursts may require queues, spare capacity, rate limits, or load shedding.
Code Example
Web tier:
3 -> 20 instances
Database:
still maxed at 100%
Result:
more callers,
same bottleneck.