Omni-Channel Ecommerce Performance Testing

We live in a digital world and a mobile society. This has changed our lives completely, even more for retailers. One bad experience, online or on mobile devices, and retail customers are gone forever. Unlike other industry domains, the retail industry is exceptionally challenged with high operational overheads and lower profit margins.

Retail applications are highly dynamic with multiple operations and modes of service. These critical applications are influencing customer service, business reputation, continuity, and growth. Bringing together various component systems like E-commerce, POS, Inventory management, and Business Intelligence systems are of the highest importance.

Retail Industry Demands More Mature Testing

When it comes to Performance Testing of any application the usual norm is to discover the objective during the course of execution which puts more stress is on the tool that any organization is going to use for the activity.

This is what we experienced during the performance testing activity for one of our clients in the retail industry. Because Royal Cyber is a premium IBM partner and  has higher level expertise on the IBM Rational Test Workbench suite, it was an obvious choice to use IBM Rational Performance Tester for this activity and we did.

Royal Cyber Performance Test Methodology

Initially, this activity was started with no clear objective except to maintain the Average Response Time for 100, 500, 1000, and 1000 users. Since it was a retail store built on IBM Commerce and has a complete backend on Commerce on Cloud, maintaining average response times was a tedious task. As all the images were getting downloaded from third-party services with caching enabled which helped to increase Response Time, especially the response time of the Home Page where all the banners are displaying, and other categories including new offers etc., along the Product Display Page.

Another, challenge faced was getting the throughput on the new relic monitoring server based on the calculation done for user’s interactions on web services. As our IBM RPT server resides in house on our local network, which has its bandwidth limitation beyond a certain number of virtual users, we were not able to get the required throughput as per the calculations based on the number of users and their direct interactions with the application.

However, we managed to find and optimize the web services and the other jQuery calls utilized on the home page, as well as the product display page to get the viable Average Response Times.

Proceeding with cloud-based solutions, like BlazeMeter, helped us get the required throughput on the new relic tool and allowed us to generate the load from dedicated geographic locations. This allowed us to imitate more realistic scenarios.

The following should be followed religiously for any Performance Testing Activity:

  • Objective (To maintain a good response time or to benchmark the application)
  • Calculation of Ramp up time and think time (Behavior of virtual users are directly proportional to think time and ramp up time, realistic calculations can only get you your desired results)
  • Realistic scenarios (Not all users should be doing only one activity, schedules should have test cases based on multiple real-time scenarios, like browsing products, checking out, or continuously adding products to the cart)
  • Server monitoring during the execution
  • Dedicated Performance Team including server-side admin (WAS specialist in our case), Senior Performance Tester, and a Developer (to take care of reading logs and interpreting exceptions).

Ready to get started with Performance Testing? Royal Cyber has strong domain exposure and profound experience in retail performance testing. We have industry knowledge and technology competencies to deliver premium service in retail software testing. For more information email us at [email protected] or visit

by Nabeel Ahmed

Leave a Reply

Live Chat