DeviQA > Case Studies > Revamping Test Automation and Performance for Solebit (MimeCast)

Revamping Test Automation and Performance for Solebit (MimeCast)

Technology Category

Application Infrastructure & Middleware - Event-Driven Application
Robots - Parallel Robots

Applicable Industries

Cement
Equipment & Machinery

Applicable Functions

Product Research & Development
Quality Assurance

Use Cases

Experimentation Automation
Time Sensitive Networking

Services

Cloud Planning, Design & Implementation Services
Testing & Certification

The Customer

Solebit

About The Customer

Solebit (MimeCast) is a cybersecurity company that specializes in the identification and prevention of zero-day malware and unknown threats. They provide solutions that protect organizations from advanced threats that bypass traditional security defenses. Their technology is designed to provide real-time protection and threat visibility, enabling organizations to effectively respond to threats and prevent data breaches. However, they were facing challenges with their testing processes, which were inefficient and time-consuming. The tests were manually triggered, took a long time to complete, and the results were difficult to interpret. The architecture of their test suite was also unscalable, making it difficult to maintain a large number of test machines.

The Challenge

Solebit (MimeCast), a company that provides identification and prevention of zero-day malware and unknown threats, was facing significant challenges with their testing processes. The existing automation tests, designed and developed by the in-house team, were not efficient. The tests were manually triggered by developers in a terminal, resulting in a large and complex file of results. The tests took over 20 hours to complete, and there was no history of the test runs available. The architecture of the test suite was unscalable, making it difficult to maintain a large number of test machines. The challenge was to create a simple runner for tests, increase their speed, redesign the architecture to support cloud platform integration, make the tests run more easily, and generate a clean report with all the necessary details. Additionally, improvements were needed for the current testing process, and some manual testing was also required.

The Solution

DeviQA created complex and detailed automated scenarios for testing REST APIs using the Faraday library. A software development kit application was built that worked on various machines, including Azure, Google Cloud, Docker, and both privileged and unprivileged LXC containers. Complex methods for these integrations were created using SSH and SFTP connections, and more than 25,000 tests were designed and developed. Each test ran 500,000 files which were then uploaded. The build time was reduced by a factor of 10, and a system was configured to run tests in a simple way without the involvement of a tech person. A reporting system was created that allowed the team to view clean reports even if the results contained 10k rows. File processing selection was configured on the number machines (Azure Cloud, Google Cloud). Autotests for uploading files were run in parallel on 30 different machines, with 10 threads on each machine. UI testing of the application was also conducted, and comprehensive test documentation was prepared. These test cases covered 80% of the application, and test suites were created for each release for different customers.

Operational Impact

The solution provided by DeviQA significantly improved the testing processes at Solebit (MimeCast). The automated scenarios for testing REST APIs were complex and detailed, ensuring thorough testing of the system. The software development kit application was versatile, working on various machines including Azure, Google Cloud, Docker, and both privileged and unprivileged LXC containers. This flexibility allowed for more comprehensive testing across different platforms. The reporting system was a significant improvement, allowing the team to view clean reports even if the results contained 10k rows. This made it easier to interpret the results and make necessary adjustments. The UI testing of the application and the comprehensive test documentation prepared also contributed to a more efficient and effective testing process.

Quantitative Benefit

Reduced the build time by a factor of 10
Configured tests to run in a simple way without the involvement of a tech person
Created a reporting system that allowed the team to view clean reports even if the results contained 10k rows