What is latency?

Latency is the time required for an operation or request to complete, usually measured from the client's or caller's perspective. It can include network time, queueing, processing, database access, serialization, and downstream service calls.

System designers should examine latency distributions such as p50, p95, and p99 rather than relying only on averages, because a small number of very slow requests can significantly affect users.