During the course of my load testing journey, I'm doing mostly spike test on the SUT(System Under Test). We gradually dial up the number of users hitting the server, from 10, 100, 1000. And we log the process execution time separately from the response time, and these are the kind of results we obtain say for a spike of 1000 users, at a single point of time:
response time: 0.1s api execution time: 0.3s
response time 0.11s api execution time: 0.4s
response time 0.15s api execution time: 0.3s
response time 0.7s api execution time: 0.5s
response time 0.9s api execution time: 0.3s
... 500th request processed by the server
response time: 7s api execution time: 0.6s
response time: 7.1s api execution time: 0.5s
response time: 7.2s api execution time: 0.4s
... 1000th request processed by the server
response time: 11s api execution time: 0.6s
response time: 11.1s api execution time: 0.5s
We speculate the increasing response time due to the queuing up of requests on the server.
As we see in the observations above, the execution time fluctuates around the same amount of time, and it's this queuing up of request that's giving us a seemingly bad response time.
So, what I'm confused on is if a spike test is actually a performance test. To me it seems more like a concurrency test in which we're trying to see if a system has some rate of failure and logging response, seems to me, doesn't have more value than a SUT on continuous expected load. These are just my ideas and I'd like more views on this point.
So, if you'd log the response times for a SUT, on a spike test and a stress test, how would you interpret those two different sets of data? And which test/test data would be better suitable to speculate and determine and benchmark the response times for a SUT?