Performance, Load & Stress Testing on Mobile Apps

First of all, it’s important to understand the key differences of these terms. It’s mostly used interchangeably and considered to be same, but actually it’s not as their goals are different.

Testing Type	Key Goals
Performance Testing	·         Establish a smooth, controlled, stable environment with a clear defined set of expectations (e.g. expected load, acceptable response time & throughput ..) & record repeatable measurements ·         Eliminate required bottlenecks, while increasing load & establish a baseline for various load levels
Load Testing	·         Is to be increase load constantly via automated tools to operate at predefined level & hence it’s part of the process of Performance testing [Incremental or Sequential] ·         Doesn’t aim to break the software, but keep the software working at the highest load possible on peak hours ·         Rectify bugs identified to ensure performance testing baseline is met
Stress Testing	·         To test the software under unpredictable scenarios like increasing load by thrice, sudden failure of DB connectivity etc and try to crash the software or see how software reacts under unexpected failures/scenarios.

Next questions that arises

1. How do we do it? What tools can help in performing this kind of testing?
2. How does performance/load/stress testing on mobile apps differ from web apps? What are the typical challenges faced at this time?                              

Starting with second Q first,

Whether it’s a mobile web or native mobile app, performing load/stress/performance testing on mobile app is similar in many ways to web app. However, testing scenarios / environment is bit different and becomes more challenging, when it comes to mobile app testing

Some of the key challenges are:

• Different types of OS (iOS, Android, Blackberry, Symbian etc) & version fragmentation
• Addressing to variety of device form factors – screen size, processor speed, multiple input modes
• Simulating bandwidth limitation of mobile network connections.
• Simulating under variety of network speeds like GPRS, EDGE, 3G, HSDPA etc
• Emulate mobile device traffic from different parts of the world
• Recording user actions performed on mobile apps
• Lack for mature test automation tools
• Load generation is bigger challenge for mobile app performance testing

There are some tools in the market that can help to some extent (as each of them have their own pros and cons OR things they can or cannot do)

• Recent annoucement by MicroFocus of launching SilkPerformer 9.0 (2012) - that supports Mobile Web & Native apps testing on iOS, Android & BB platforms
• HP LoadRunner – Best tool as of now -  that supports Mobile Web & Native apps testing on iOS, Android, BB & WM platforms
• Use NeoLoad that supports Mobile Web & Native Mobile Apps
• Use JMeter for Mobile Web App Testing (Simulators)
• IBM Rational Performance Tester (RPT) for Mobile Web App Testing

Hope to see some new tools coming soon and make life easy for Mobile QA testers!