Neptune.ai

Brainly, a leading global learning platform, faced a challenge with their machine learning-powered feature, Snap to Solve. The feature allows users to upload a photo of a problem, which the system then detects and provides solutions for. The Visual Search team, responsible for the Visual Content Extraction (VICE) system of Snap to Solve, used Amazon SageMaker to run their computing workloads and serve their models. However, as the number of training runs on their large compute architectures increased, they found that their logs from Amazon SageMaker needed to be trackable and manageable to avoid workflow bottlenecks. They needed a tool that could scale regardless of the experiment volume. While they tried using SageMaker Experiments for tracking, they found the tracking UX and Python client unsatisfactory.

Theta Tech AI, a company that builds customized artificial intelligence algorithms and front-end user interfaces for large-scale healthcare AI systems, faced several challenges in developing generalizable medical AI systems. The team had to manage thousands of experiments for large-scale parallel training workflows, which were run on GPU servers in AWS. However, they found that AWS CloudWatch Logs, their initial choice for monitoring the jobs, was inadequate for managing experiment logs. The team was unable to get experiment-relevant metrics from AWS CloudWatch Logs, debug problems with training jobs and experiments, integrate Optuna for hyperparameter optimization, and communicate the results of ML models to clients effectively.

Continuum Industries, a company in the infrastructure industry, developed a product called Optioneer to automate and optimize the design of linear infrastructure assets. However, operating the Optioneer engine presented several challenges. The engine needed to be robust enough to handle different problems fed by different customers. Moreover, the company needed to ensure that the solutions provided by the engine were accurate and agreed upon by civil engineers. The team also had to constantly improve the optimization engine without breaking the algorithm. The nature of the problem they were trying to solve presented additional challenges. They could not automatically tell whether an algorithm output was correct or not. They needed a set of example problems that was representative of the kind of problem that the algorithm would be asked to solve in production. The team initially developed a custom solution to these problems, but it proved to be extremely clunky and complex to maintain.