Things to check when you cannot replicate an end user performance problem

Here are a few things to consider when confronted with users complaining about poor performance and you cannot replicate the issue with your configuration. This is looking at comparing your tests verses a users. Most of the points consider the set up on the end users device.

1) Same Hardware/Software? OK may sound obvious but often over looked particularly around the details of software versions.

2) Are you doing a like for like transaction. I once heard of a support call that complained that it took ages to find a car part when doing a search on the inventory system. It was later discovered that the user was doing a wild card search against the inventory and then scroll through the results to find the part!

3) Same start and end points? Users often think in completing processes rather than transactions. When in doubt ask the users to screenshot the start and end points. For windows systems they can use Microsoft Step Recorder.

4) Do you have the same data sets? Your test user account on the system is most likely not to have the history of claims/sales/orders of a real users.

5) Security settings? This is common when support staff are using different desktop builds to the customer. Common differences beyond the hardware differences are encrypted disks and exclusion rules for anti-virus

6) Browser version and setting? There is a significant difference in performance across browser types and versions. Always check you are testing like for like. Plus don’t forget any plugins.

7) Connectivity? I have even seen problems with two users side be side having different connectivity characteristics even on the same subnet. The problem was one user has wifi enabled and it would connect to there wifi hotspot on their phone! rather than the corporate network. More importantly end users will have difference latency and bandwidth availably than you. Browsers like chrome allow you to emulate some network characteristics.

8) Observer effect? Your debugging tools on your machine may change the outcome of the test. I have seen this with debugging proxies like Fiddler. No please don’t use this as a reason not to use those tool, life would be very difficult without them! Also, don’t expect to RDP into the machine and the results to the best same here is an example RDP example

9) Screen size can make difference, I once had a system where users with larger displays has some transaction take longer as it was rendering more items to the screen for each transaction.

10) Background processes can consume CPU and network bandwidth that can impact the end users machine performance.

Author: loadtestguy

Just a performance tester that wants to record the things I should remember about LoadTesting

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: