DeviQA Case Studies Revamping Test Automation and Performance for Solebit (MimeCast)

	Revamping Test Automation and Performance for Solebit (MimeCast) DeviQA

Revamping Test Automation and Performance for Solebit (MimeCast)

DeviQA

Technology Category	Application Infrastructure & Middleware - Event-Driven Application Robots - Parallel Robots
Applicable Industries	Cement Equipment & Machinery
Applicable Functions	Product Research & Development Quality Assurance
Use Cases	Experimentation Automation Time Sensitive Networking
Services	Cloud Planning, Design & Implementation Services Testing & Certification
Challenge	Solebit (MimeCast), a company that provides identification and prevention of zero-day malware and unknown threats, was facing significant challenges with their testing processes. The existing automation tests, designed and developed by the in-house team, were not efficient. The tests were manually triggered by developers in a terminal, resulting in a large and complex file of results. The tests took over 20 hours to complete, and there was no history of the test runs available. The architecture of the test suite was unscalable, making it difficult to maintain a large number of test machines. The challenge was to create a simple runner for tests, increase their speed, redesign the architecture to support cloud platform integration, make the tests run more easily, and generate a clean report with all the necessary details. Additionally, improvements were needed for the current testing process, and some manual testing was also required. Read More
About Customer	Solebit (MimeCast) is a cybersecurity company that specializes in the identification and prevention of zero-day malware and unknown threats. They provide solutions that protect organizations from advanced threats that bypass traditional security defenses. Their technology is designed to provide real-time protection and threat visibility, enabling organizations to effectively respond to threats and prevent data breaches. However, they were facing challenges with their testing processes, which were inefficient and time-consuming. The tests were manually triggered, took a long time to complete, and the results were difficult to interpret. The architecture of their test suite was also unscalable, making it difficult to maintain a large number of test machines. Read More
Customer Name	Solebit Read More
Solution	DeviQA created complex and detailed automated scenarios for testing REST APIs using the Faraday library. A software development kit application was built that worked on various machines, including Azure, Google Cloud, Docker, and both privileged and unprivileged LXC containers. Complex methods for these integrations were created using SSH and SFTP connections, and more than 25,000 tests were designed and developed. Each test ran 500,000 files which were then uploaded. The build time was reduced by a factor of 10, and a system was configured to run tests in a simple way without the involvement of a tech person. A reporting system was created that allowed the team to view clean reports even if the results contained 10k rows. File processing selection was configured on the number machines (Azure Cloud, Google Cloud). Autotests for uploading files were run in parallel on 30 different machines, with 10 threads on each machine. UI testing of the application was also conducted, and comprehensive test documentation was prepared. These test cases covered 80% of the application, and test suites were created for each release for different customers. Read More Log in to view content
Contents

Technology Category

Application Infrastructure & Middleware - Event-Driven Application

Robots - Parallel Robots

Applicable Industries

Cement

Equipment & Machinery

Applicable Functions

Product Research & Development

Quality Assurance

Use Cases

Experimentation Automation

Time Sensitive Networking

Services

Cloud Planning, Design & Implementation Services

Testing & Certification

Challenge

Solebit (MimeCast), a company that provides identification and prevention of zero-day malware and unknown threats, was facing significant challenges with their testing processes. The existing automation tests, designed and developed by the in-house team, were not efficient. The tests were manually triggered by developers in a terminal, resulting in a large and complex file of results. The tests took over 20 hours to complete, and there was no history of the test runs available. The architecture of the test suite was unscalable, making it difficult to maintain a large number of test machines. The challenge was to create a simple runner for tests, increase their speed, redesign the architecture to support cloud platform integration, make the tests run more easily, and generate a clean report with all the necessary details. Additionally, improvements were needed for the current testing process, and some manual testing was also required.

About Customer

Solebit (MimeCast) is a cybersecurity company that specializes in the identification and prevention of zero-day malware and unknown threats. They provide solutions that protect organizations from advanced threats that bypass traditional security defenses. Their technology is designed to provide real-time protection and threat visibility, enabling organizations to effectively respond to threats and prevent data breaches. However, they were facing challenges with their testing processes, which were inefficient and time-consuming. The tests were manually triggered, took a long time to complete, and the results were difficult to interpret. The architecture of their test suite was also unscalable, making it difficult to maintain a large number of test machines.

Customer Name

Solebit

Solution

DeviQA created complex and detailed automated scenarios for testing REST APIs using the Faraday library. A software development kit application was built that worked on various machines, including Azure, Google Cloud, Docker, and both privileged and unprivileged LXC containers. Complex methods for these integrations were created using SSH and SFTP connections, and more than 25,000 tests were designed and developed. Each test ran 500,000 files which were then uploaded. The build time was reduced by a factor of 10, and a system was configured to run tests in a simple way without the involvement of a tech person. A reporting system was created that allowed the team to view clean reports even if the results contained 10k rows. File processing selection was configured on the number machines (Azure Cloud, Google Cloud). Autotests for uploading files were run in parallel on 30 different machines, with 10 threads on each machine. UI testing of the application was also conducted, and comprehensive test documentation was prepared. These test cases covered 80% of the application, and test suites were created for each release for different customers.

Impact #1	The solution provided by DeviQA significantly improved the testing processes at Solebit (MimeCast). The automated scenarios for testing REST APIs were complex and detailed, ensuring thorough testing of the system. The software development kit application was versatile, working on various machines including Azure, Google Cloud, Docker, and both privileged and unprivileged LXC containers. This flexibility allowed for more comprehensive testing across different platforms. The reporting system was a significant improvement, allowing the team to view clean reports even if the results contained 10k rows. This made it easier to interpret the results and make necessary adjustments. The UI testing of the application and the comprehensive test documentation prepared also contributed to a more efficient and effective testing process.

Impact #1

The solution provided by DeviQA significantly improved the testing processes at Solebit (MimeCast). The automated scenarios for testing REST APIs were complex and detailed, ensuring thorough testing of the system. The software development kit application was versatile, working on various machines including Azure, Google Cloud, Docker, and both privileged and unprivileged LXC containers. This flexibility allowed for more comprehensive testing across different platforms. The reporting system was a significant improvement, allowing the team to view clean reports even if the results contained 10k rows. This made it easier to interpret the results and make necessary adjustments. The UI testing of the application and the comprehensive test documentation prepared also contributed to a more efficient and effective testing process.

Benefit #1	Reduced the build time by a factor of 10
Benefit #2	Configured tests to run in a simple way without the involvement of a tech person
Benefit #3	Created a reporting system that allowed the team to view clean reports even if the results contained 10k rows

Benefit #1

Reduced the build time by a factor of 10

Benefit #2

Configured tests to run in a simple way without the involvement of a tech person

Benefit #3

Created a reporting system that allowed the team to view clean reports even if the results contained 10k rows

Download PDF Version

Overview

Revamping Test Automation and Performance for Solebit (MimeCast)

Operational Impact

Quantitative Benefit