Measuring Web Application Performance – 1/3

During a typical lunch talk, a colleague said that we could easily list the most popular stereotypes about a nation just by asking  Google:

After some laughs I started wondering about public opinion concerns on the main web sites of the internet. Adapting a little the previous question I got some interesting answers:

I was surprised to see that the results indicated that most people were asking about the same topic:  web sites performance

So it looks like the speed at which the content is delivered is as important as the content it-self. “Subconsciously, you don’t like to wait,” said Arvind Jain, a Google engineer who is the company’s resident Speed maestro. “Every millisecond matters.”

According to the New York Times:

Google and other tech companies are on a new quest for speed, challenging the likes of Mr. Jain to make fast go faster. The reason is that data-hungry smartphones and tablets are creating frustrating digital traffic jams, as people download maps, video clips of sports highlights, news updates or recommendations for nearby restaurants. The competition to be the quickest is fierce

A simple comparison would be a supermarket right next to your house and another on the other side of town, you will of course go to the closer one since it takes you less time to do your shopping. In other words, people tend to visit sites that are proving content faster than competitors. And this raises the question: "How faster than my competitors should I be?".

Quote from Harry Shum, a computer scientist and speed specialist at Microsoft:

Two hundred fifty milliseconds, either slower or faster, is close to the magic number now for competitive advantage on the Web

250 milliseconds (1/4 of a second, or literally a wink of an eye) is considered the number to be faster than your competitor. In 2009, a study by Forrester Research found that online buyers have lot of money and little patience. They expect pages to load within  two seconds or less. Between three and four seconds they have a dissatisfied customers (and unhappy customers tend not to do many purchases). After 5 seconds they just leave.

Just three years earlier, a similar study from Forrester found that the average expectations for page loading times were of 4 seconds or less,  so it means that increasingly the customer who is spending money on the web wants to be answered quickly.

Time is money

Surveys conducted by Amazon demonstrated a 100 ms increased loading time results in 1% loss in sales (source). In this case, how much cash would lose a virtual store that happens to be 1 second longer to load ?

• 100 milliseconds = -1% of sales.
• 1 second = 1000 milliseconds.
• 1 second = could mean until 10% of sales.

Imagine how would react the owner of a virtual store after hearing that he loses 10% of potential sales every second that his Web site slows down?

The one million dollar question

Can you see the paradox? All companies want their web site to have a growing volume of visitors, however the higher the number of visitors, the higher the server's response time and a slower loading.

We could came to the conclusion that the more traffic your have the slower you pages will load and then the less the money.

With this principle in mind, are you aware of the load time threshold that makes a user stay or leave your web site ?

What is making your web site so slow? It's easy just do some performance tests on your application and you will probably get the right answer.

Performance testing consists in evaluating the capacity, robustness and availability of an application versus the amount of concurrent connections, evaluating its performance especially under high workload and also considering its behavior under normal circumstances.

The main goal of such experiments is to ensure that the software does not present problems or outages in conditions of insufficient computational resources (such as memory, disk space or processing), when working in high concurrency or suffering some denial-of-service attack. Some benefits of performance testing are:

• Check the quality of the system developed
• Test the infrastructure capacity hired
• Be aware of supported simultaneous access
• Identify the point of bottlenecks of its implementation, the performance SPOF

There are some techniques to evaluate the performance of a web application, the top 3 are:

• Load test
• Stress testing
• Non-regression test of Performance.

Load test

During a load test the number of simultaneous connections is growing over time. The goal is to evaluate the application's behavior in accordance with the growth of traffic (requests) so that we can identify both application's capacity limit and bottleneck (coding, hardware, database ...)

Stress Test

In a stress test, a higher load beyond normal usage patterns is fired against the application, some crash scenarios are tested, in order to determine the resilience and stability of the system. For instance this could be made by removing a node from the clusters of application servers or making a database unavailable.

Non-regression Performance Test

Ensures that a modification of the system has no impact on its performance.

How does the load test work?

According to Wikipedia: load test is used to check the limit of data processed by the software until it can't not process it. It is also known as Volume test:

Used to validate and assess the acceptability of the operational limits of a system according to varying workloads, whereas the system under test remains constant. In General, measurements are taken on the basis of the data transfer rate of the workload and transaction response time aligned

The goal of the load test is to determine the Web application's behavior through normal conditions and high peak loads. As they gradually increments from normal load to peak,  response time and hardware behavior should be analysed.

How to identify application's limit?

To get to know your application's limit, an evaluation should be made on four aspects:

• Unavailability
• High response time
• Data inconsistency
• Concurrency issues

Looking at the figure below, we have a simple example of run report, where we can see the results of a 18 minutes test , having a limit of 1,300 concurrent users, with a ramp up  of 100 users per minute.

In other words, imagine that your virtual store made a huge promotion ant that  each minute 100 new users are connecting. In this scenario, how many users can you application serve without noticeable impact? What is the number of users and the time threshold that turns your application into a slow website that users just want to leave ? Analyzing the graph generated in this test, we can say that till 8 minutes, or approximately 800 concurrent users, the application behaved stably, but after that we had a high latency and a high response time and so we identify our website stress limit point.

In the next article we will see how a load test must be planned and executed to help us being aware of  our web application performance