Latency is one of the most important metrics for measuring the performance of an application or service. It represents the amount of time it takes for a request to be processed—from the moment a client sends the request until it receives the response.
When monitoring applications, engineers rarely rely only on the average latency because averages can hide slow requests. Instead, modern monitoring systems report latency percentiles, such as P50, P95, and P99, which provide a much more accurate picture of the user experience.
This document explains what latency percentiles are, how they are calculated, and why they are essential for monitoring production systems.
A percentile tells us the value below which a certain percentage of observations fall.
For latency measurements:
P50 means 50% of requests completed in that amount of time or less.
P95 means 95% of requests completed in that amount of time or less.
P99 means 99% of requests completed in that amount of time or less.
To calculate percentiles, all request durations are first sorted from the fastest to the slowest.
For example, suppose an API processes 10,000 requests in one hour. After sorting the response times, monitoring tools determine the latency values corresponding to the 50th, 95th, and 99th percentiles.
The 50th percentile (P50) is also known as the median.
If:
P50 = 35 ms
then:
50% of requests completed in 35 ms or less
The remaining 50% took longer than 35 ms
P50 represents the typical user experience, since half of all users receive responses faster than this value.
The 95th percentile (P95) measures the experience of almost all users.
Imagine your application processes 100 requests.
After sorting the response times:
RequestResponse Time110 ms212 ms......90120 ms91130 ms92140 ms93150 ms94160 ms95180 ms96220 ms97260 ms98350 ms99750 ms1002000 ms
The 95th percentile is 180 ms.
This means:
95 out of 100 requests completed in 180 ms or less
Only the slowest 5 requests took longer than 180 ms
Notice that the requests taking 220 ms, 260 ms, 350 ms, 750 ms, and 2000 ms are not included in the P95 value because they belong to the slowest 5%.
Suppose your service processes 10,000 requests.
If:
P95 = 180 ms
then:
9,500 requests completed in 180 ms or less
500 requests took longer than 180 ms
This does not mean every request took 180 ms.
It simply means that 95% of all requests completed within 180 milliseconds.
The 99th percentile (P99) focuses on the slowest requests and helps identify rare performance problems.
Consider the same 100 requests:
RequestResponse Time110 ms212 ms......95180 ms96220 ms97260 ms98350 ms99750 ms1002000 ms
The 99th percentile is 750 ms.
This means:
99 out of 100 requests completed in 750 ms or less
Only 1 request took longer than 750 ms
Notice that the slowest request (2000 ms) is not the P99 value. It belongs to the remaining 1% of requests that exceeded the 99th percentile.
Suppose your application processes 10,000 requests.
If:
P99 = 600 ms
then:
9,900 requests completed in 600 ms or less
100 requests took longer than 600 ms
Again, this does not mean every request took 600 ms.
It simply means that 99% of requests finished within 600 milliseconds.
Imagine the sorted response times below:
10 12 14 15 16 18 20 ... 30 35 40 60 80 120 180 250 400 750 900 1200
A monitoring system might report:
MetricValueMinimum10 msAverage42 msP5035 msP95180 msP99750 msMaximum1200 ms
This tells us:
Most users receive responses within 35 ms.
Nearly every user receives a response within 180 ms.
Almost all users receive a response within 750 ms.
Only a very small number of requests exceed 750 ms.
The slowest request took 1200 ms.
Average latency can be misleading because a small number of very slow requests may have little impact on the average.
Consider two services.
99 requests = 20 ms
1 request = 2000 ms
Average latency:
39.8 ms
Although the average looks good, one user experienced a delay of 2 seconds.
100 requests = 40 ms
Average latency:
40 ms
The average is almost identical to Service A, yet every user receives a consistent experience.
This demonstrates why engineers rely on percentiles instead of averages.
Both P95 and P99 provide insights that average latency cannot.
P95 represents the experience of almost all users.
It is commonly used for:
Measuring API performance
Monitoring web applications
Defining Service Level Objectives (SLOs)
Tracking customer experience
Because P95 ignores only the slowest 5% of requests, it provides a stable measurement of overall performance while filtering out extreme outliers.
P99 focuses on the slowest users and is often called tail latency.
It is especially useful for detecting intermittent performance issues such as:
Database locks
Network latency spikes
Garbage collection pauses
Thread starvation
High CPU utilization
Resource contention
Infrastructure problems
P99 is much more sensitive to performance spikes than P95.
Even if only 1% of users are affected, those requests often reveal problems that deserve investigation.
Suppose your monitoring dashboard reports:
MetricValueAverage55 msP95140 msP99820 msMaximum4200 ms
Most requests complete quickly (around 55 ms on average).
95% of users receive responses within 140 ms.
99% of users receive responses within 820 ms.
About 1% of requests experience noticeable delays.
The slowest request took 4.2 seconds, indicating an occasional performance problem that should be investigated.
Each latency metric answers a different question.
MetricMeaningWhat It Tells YouMinimumFastest requestBest-case latencyAverageMean response timeOverall performance, but can hide slow requestsP50Median latencyTypical user experienceP9595% of requests are faster than this valueExperience of almost all usersP9999% of requests are faster than this valuePerformance of the slowest requests (tail latency)MaximumSlowest requestWorst-case latency
When monitoring production systems:
Never rely solely on average latency.
Monitor P50 to understand the typical user experience.
Monitor P95 to ensure that almost all users receive acceptable performance.
Monitor P99 to detect rare but significant performance issues.
Investigate increases in P99 even if the average latency remains unchanged, as this often indicates intermittent bottlenecks.
Latency measures how long it takes to process a request.
Percentiles provide a much better representation of user experience than averages.
P50 represents the typical request.
P95 represents the experience of almost all users.
P99 highlights rare but slow requests and is essential for identifying tail latency.
Average latency can hide serious performance problems, whereas P95 and P99 reveal them.
Successful production monitoring relies on tracking percentile metrics alongside averages to gain a complete understanding of application performance.
