Sample questions
Pf Metrics Percentiles AnalysisDifficulty 1
In a performance test report, why is the p95 response time usually more useful than the average (mean) response time?
- aBecause the p95 value is always mathematically smaller than the average
- bBecause a small number of very slow requests can pull the average up or down without showing how the slowest users actually experienced the system, while p95 directly reports a tail value✓
- cBecause calculating an arithmetic average requires the reporting tool to keep every single raw sample sorted in memory for the whole test duration, which costs far more CPU and memory than deriving a percentile from a compact histogram structure
- dBecause averages can only be computed when the sample size is an even number, so odd-sized test runs must fall back to reporting a percentile instead
Explanation:A handful of very slow (or very fast) outliers can distort the arithmetic mean, hiding how bad the experience is for the slower fraction of users. p95 is a tail-focused value: it directly tells you the response time below which 95% of requests fell, which is closer to what a real user's 'worst normal' experience looks like.
Pf Metrics Percentiles AnalysisDifficulty 1
What does the p50 (50th percentile) of a response-time distribution represent?
- aThe slowest response time recorded during the test
- bThe average of the fastest and slowest response times
- cThe response time that exactly 50 requests exceeded, regardless of total sample size
- dThe median: the value below which 50% of the recorded response times fall✓
Explanation:p50 is simply the median of the distribution — the point at which half of the observed response times are lower and half are higher. It is not tied to a fixed count of 50 requests; it always represents 50% of whatever the total sample size is.
Pf Metrics Percentiles AnalysisDifficulty 1
If a load test report shows p99 = 2400 ms for an endpoint, what does this number mean?
- a99% of the recorded requests completed in 2400 ms or less, and 1% took longer✓
- b99% of the requests failed and only 1% succeeded in 2400 ms
- cThe test ran for 99% of 2400 ms before stopping
- d2400 ms is the exact response time of the 99th request sent during the test
Explanation:p99 = 2400 ms means that when you sort all recorded response times, 99% of them are at or below 2400 ms, leaving the slowest 1% above it. It has nothing to do with failure rate, elapsed test duration, or a specific request's ordinal position.
Pf Metrics Percentiles AnalysisDifficulty 2
Why do performance engineers often care more about p99 than p50 when evaluating a large-scale production API?
- aBecause p99 is always easier to compute than p50
- bBecause p99 always equals the maximum recorded response time
- cBecause at high request volumes, even a 'rare' 1% tail translates into a large absolute number of real users experiencing slow responses✓
- dBecause monitoring tools cannot calculate p50 for endpoints that receive more than 1000 requests in a single test run, so they silently fall back to reporting only the maximum value instead
Explanation:At scale, percentages translate into real user counts: if an API serves 1,000,000 requests, the 1% behind p99 is 10,000 slow requests — not a negligible edge case. p50 hides this entirely, since it only reflects the 'typical' middle of the distribution.
Pf Metrics Percentiles AnalysisDifficulty 1
In general statistics terms, what is a percentile?
- aA fixed unit of time such as a millisecond or second
- bA value below which a given percentage of the observations in a data set fall✓
- cThe square root of the variance of a data set
- dA count of how many times a specific value repeats in a data set
Explanation:A percentile is a statistical measure indicating the value below which a certain percentage of observations in a dataset fall. p95 is the value below which 95% of the data points lie.
Pf Metrics Percentiles AnalysisDifficulty 2
A team reports 'average response time: 220 ms' for an endpoint and declares performance healthy. Later they discover that 60% of requests actually complete in 80 ms while the remaining 40% complete in 450 ms (a bimodal distribution, e.g. cache hits vs cache misses). What does this scenario illustrate?
- aThat the average was calculated incorrectly and should be recalculated using a different formula
- bThat bimodal distributions always mean the load test tool has a bug
- cThat a single average number can mask two very different sub-populations of requests, giving a misleadingly 'moderate' picture of performance✓
- dThat cache misses should always be filtered out and excluded from performance test results entirely, since they represent an abnormal condition rather than real production traffic
Explanation:The average (220 ms) sits between the two clusters (80 ms and 450 ms) and does not represent either group well — no individual request actually experiences ~220 ms. This is the classic pitfall of averaging a bimodal distribution: the summary number looks moderate while nearly half the traffic is actually slow.