.NET Architecture & System Design Interview Question #188

Why can autoscaling fail to solve an overloaded system?

Deployment, Cloud Operations, SRE & Senior System Design Scenarios Senior Advanced

Quick Interview Answer

Autoscaling one application tier does not create capacity in its dependencies. Adding web instances can increase pressure on a database, external API, connection pool, or shared cache and make the actual bottleneck worse.

Detailed Explanation

Autoscaling one application tier does not create capacity in its dependencies. Adding web instances can increase pressure on a database, external API, connection pool, or shared cache and make the actual bottleneck worse.

Architecture should identify the constrained resource and scale the system end to end. Autoscaling also reacts with delay, so sudden bursts may require queues, spare capacity, rate limits, or load shedding.

Code Example

Web tier:
3 -> 20 instances

Database:
still maxed at 100%

Result:
more callers,
same bottleneck.