Author: Amir Rozenberg, Sr. Director Mobile Performance and Monitoring Products, Perfecto Mobile
The team had spent enormous cycles creating a new application. It has been tested in the various QA contexts: functional, automation and performance. The marketing team has done a tremendous job promoting the new application launch and the audience is anticipating its arrival into the app store. On launch day, user downloads, engagement times, product lookup, add to cart and checkouts are exceeding all expectations. Everyone’s happy.
On the 3rd day there’s a user review on the app store about the app crashing while adding the product to the cart. From then on, more and more reviews in the store, Facebook, twitter and other areas talk about app crashes, delays, and other frustrating experiences. As it turns, once in a while the application logic would not pick up a condition and would go into an error state. Combine that with an intermittent issue on the load balancer, redirecting mobile requests to the non-mobile server. Not before long, users start to remove the applications from their devices and despite enormous efforts by the team to produce a new, stable version, the magic is gone and the user adoption is no longer there.
Fig1: Users reaction to poor experience (Source)
It’s the nightmare of the mobile application owner. The operations team was using the traditional web monitoring tools, which all indicated green light, where in reality users were becoming increasingly frustrated. What’s the gap?
The gap is that a mobile experience does not equal web experiences. Web experiences are very much about network traffic, which is a close reflection to what the end user experience is. That is not the case in mobile; the mobile device is managing the network traffic, but it does much more:
- It processes content from all back ends including 3rd parties
- It authenticates the user
- encrypts the data
- Utilizing native resources (GPS, NFC, Camera etc.)
- Handling the application UI
The end user experience is the sum of all of these components, and it is dependent on the least performing bottleneck.
As a prelude to several blog posts that will follow, we wanted to give insight to some of the best practices that facilitate an applicable mobile monitoring solution:
- Synthetic mobile monitoring is complementary to Real User Monitoring: The synthetic monitoring objective is to get as close to the real user experience, while facilitating efficient triage and isolation process. The solution should provide all insight required to follow up with root cause analysis, whether it’s response time and availability of key transactions, device vitals, network traffic and samples from the UI.
- Leverage real devices, drive the test like the user and measure the results the way users would experience those: Leverage non-jailborken/rooted devices, non-modified apps (use the one in production), drive the test on the device like your users would and measure the transactions based on what users experience, the UI.
- Expand existing investment in your monitoring solution to become mobile-relevant: Your existing run operations center is made of years of experience monitoring and triaging incidents. It makes no sense and there is no need to create new ones, buy completely new solutions etc. Rather, expand those processes to cover your mobile initiative as well by making it mobile-relevant.
- Select your synthetic monitoring coverage wisely: Synthetic monitoring, by definition, provides sampled coverage of the audience, devices, carriers and locations, and user scenarios. It is not realistic to provide coverage for everything. The task is to identify those combinations that are relevant to your success criteria and drive it from there.
- Ensure your monitoring solution to provide high-SLA and genuine alerts: Your mobile monitoring solution has to provide high SLA so that your operation center engineers need not waste time on alerts caused by monitoring device availability, as example. The single most frustrating item for these engineers are false alerts. Device redundancy, identification and reduction of alerts driven from known issues are key to ensure proper attention to real issues that are impacting end users.
Bottom Line
Mobile monitoring is a relatively new practice, which introduces a new set of challenges to the operations team. Devices, as example, are consumer-facing devices and are not as stable as servers would in an ongoing monitoring scenario. The objective is to build a solution that would overcome these challenges, address the sheer number of new devices available on the market in such a frequent basis, and provide relevant insight both to the operations center staff as well as the application owner.
To read more on the best practices of real device monitoring, please refer to the white-paper develop by Perfecto Mobile and is available here.