Case Study: Enhancing Product Quality + Efficiency with QA Automation

The Challenge

This global electronics provider required a partner to help implement an integrated test automation solution within the development process of its eCommerce application. The solution needed to include functional tests for both web and service components and a suite of performance tests to ensure that any changes made would not impact response times. The solution had to be developed from scratch, as there was no existing foundation for it.

At the start of the partnership with Gorilla Logic, the client was also working with another development vendor to help its teams execute several projects. Due to many positive relationships and successful outcomes with the Gorilla Logic team, the client eventually chose to work exclusively with Gorilla Logic, leading to the termination of services provided by the other vendor. 

The client selected Gorilla Logic to help them:

• Implement an automation solution to verify functional tests for both the web and service components
• Ensure compatibility of the new automation solution with the existing infrastructure and architecture of the eCommerce system
• Establish seamless integration of new automation processes into existing development process

The Gorilla Logic Approach

Working with the client’s internal teams, Gorilla Logic divided the existing system into modules, each with a dedicated team in charge of implementing the QA automation solution. An additional team was responsible for the overall system infrastructure and architecture. 

Gorilla Logic initiated the analysis for implementing the test automation solution, defined the QA process, and constructed several artifacts including test cases, defects, and test plans.

Key deliverables:

• Developed a web test automation framework to validate the web component of the system using Protractor, JavaScript, Mocha, Jira Integration, Influx, Grafana, and Concourse.
• Developed an automation framework to validate approximately 25 services using Chai HTTP, JavaScript, Mocha, Jira Integration, Influx, Grafana, and Concourse.
• Developed a performance suite focusing on the most critical services using JMeter, Influx, Grafana, and Concourse.
• Created a collection in Postman of all services to enable more accurate troubleshooting of potential errors and quicker testing.
• Migrated the database engine from MongoDB to Cosmos DB. 

Using these frameworks and testing suites, Gorilla Logic conducted automated tests on both web and service components daily (and on-demand) in various environments. The results from the tests were accessible to the client’s internal teams in Concourse and were also stored in an Influx database. Gorilla Logic used the data from the automated tests to generate dynamic reports in Grafana, providing the client with insights into the percentage of automated tests, time saved in comparison to manual testing, and other key performance indicators. 

Before and after deployment, Gorilla Logic also conducted performance tests, helping them easily identify any changes that impacted the service's performance. The results from the performance tests were sent to the Influx database, and a comparison report in Grafana displayed the results. 

Business Outcomes

The QA automation solution implemented by Gorilla Logic provided multiple benefits for the client, including a significant reduction in time spent on testing and a higher-quality product overall. 

• Decreased the time required for sanity testing from 15 hours per week to just 5 hours per week–a 66% time reduction. 
• Decreased the time required for regression testing (manual and automated) from 2 weeks to 1 week–a 50% time reduction. 
• Expanded the client’s capacity for testing and other development work due to the efficiency of new QA processes.