A ‘how-to’ guide for test automation framework with Allure Reporting

The software industry has seen one of the most dramatic changes with the improved methodologies, Open-Source collaboration, DevOps, and migration to the Cloud. This has in turn resulted in a reduction in time to market an application from a few years to months. To keep up with these advancements and to deliver bug free software it has become essential to utilize Test Automation in Quality Assurance practice. 

To keep pace with swift development, we needed a test framework which able to handle all the current and future Quality Assurance requirements for Web Applications and microservices, routinely carry out testing/monitoring of individual Product Dashboards which were deployed across multiple environments and lastly should be able to provide prompt notification whenever any issues were observed. 

What were we going to achieve with test automation? 

A robust Automation framework which is scalable to handle all the Quality Assurance needs for Web Applications and microservices. With primary focus on the below criteria were taken into account for selection of framework: 

● Reusability of code. 

● Low maintenance. 

● No manual intervention. 

● Accuracy and reliability. 

● Email notifications. 

● Ability to cover e2e scenarios. 

● Concise reporting and precise logging. 

● Monitoring dashboard. 

● Future proof. 

● Platform independence. 

● Easy collaboration through version control. 

Design: 

The following diagram shows the Automation framework structure utilized to meet up with the above requirements:

JAVA: Most used programming language for test automation. Selenium and REST Assured(Open Source API testing library only for JAVA ) work best with JAVA. Other extensive libraries are available for most QA requirements. MAVEN: Build automation / Dependency management. 

ALLURE: Reporting Tool. 

Libraries: 

TestNG: Testing framework for JAVA. With ability to cover a wider range of test categories: unit, functional, end-to-end, integration, etc., with many powerful and easy-to-use functionalities. 

REST Assured: Open-Source JAVA library to automate API Testing 

Selenium: Open-Source library for web browser automation. 

Test Infrastructure setup: 

To automate and trigger the build process, the below test infrastructure was utilized.

JENKINS: Automation Server. 

GIT: Version Control 

Allure Report: Reporting Dashboard and logging 

Apache Web Server: Hosting reporting dashboard. 

Automation Approach: 

● Open Source: 

● Same Test Infrastructure for both API and UI Testing. 

● Cross browser testing. 

● Code reusability and easy to incorporate multiple environments. 

● Effort saving: The POM and PageFactory coding standard for Selenium makes it easy to maintain. Automated API request sending part to reduce coding required. 

● Reporting Dashboard: Test Execution Report can be hosted on the AWS EC2 instance by utilizing Apache Web Server and made available to all the project stakeholders. For further security, the Reporting dashboard can be password protected.

● Email Notification: 

Scripting has been done to create a custom Email HTML template which contains concise Test reports with relevant information. This email is further shared with appropriate stakeholders. 

Email notifications for different projects:

Conclusion: 

1. Automated and replaced the manual effort required in daily monitoring of multiple dashboards used by customers. This enabled the developers to act on any observed issues before customers. 

2. Business team is able to highlight any discrepancies in advance and provide prompt response to the same thus improving customer experience. 

3. Effort saving for Developers in reproducing issues owing to error screenshots and detailed logging in the Reporting Dashboard. 

4. The developed framework is scalable and should be able to handle additional customer dashboards with minimal efforts. 

Looking to authenticate the allure dashboard? 

Setup and secure the reporting dashboard.