Edge Delta Case Studies Fama's Journey to 100% Datadog Visibility and Anomaly Detection Automation with Edge Delta

Edit This Case Study Record

	Fama's Journey to 100% Datadog Visibility and Anomaly Detection Automation with Edge Delta Edge Delta

Fama's Journey to 100% Datadog Visibility and Anomaly Detection Automation with Edge Delta

Edge Delta

Technology Category	Analytics & Modeling - Machine Learning Robots - Parallel Robots
Applicable Industries	Buildings Construction & Infrastructure
Applicable Functions	Maintenance
Use Cases	Intrusion Detection Systems Supply Chain Visibility
Challenge	Fama, a leading background screening company, was facing a significant challenge with its Datadog adoption. The company had recently modernized its observability stack and moved away from a self-hosted log management platform to Datadog. However, the costs associated with Datadog were much higher than projected, with Fama spending between 2-3x each month as much as the team had forecasted. This was due to two factors: Fama's growth and the size of their logs. The company had doubled its screening volume from 2020 to 2021, naturally creating more logs. Additionally, their logs were larger than anticipated, with what should have been single log events often being three or four events. Attempts to strategize ways to lower costs with Datadog either didn't make enough of a difference or created significant blindspots by filtering out logs. Fama needed a solution that would reduce the volume of data indexed into Datadog without sacrificing visibility. Read More
About Customer	Founded in 2016, Fama is a leading background screening company that helps organizations make better hiring decisions by scraping a candidate’s public social media presence and using machine learning to identify problematic behaviors. The company supports organizations around the world, including a majority of the largest executive search firms. Fama is always looking for new, cutting-edge technologies to better support its customers. One of its core initiatives has been to streamline its infrastructure and operations by adopting a greenfield serverless architecture on AWS. The company’s upcoming releases will run solely on this serverless architecture. Read More
Solution	Fama found its solution in Edge Delta, an observability platform that complemented and integrated with their existing Datadog implementation. Edge Delta provided out-of-the-box integrations to support Fama's serverless adoption, namely with Amazon CloudWatch and AWS Lambda. It also addressed Fama's core cost challenge without forcing them to sacrifice visibility through sampling or filtering logs. Edge Delta analyzes raw datasets as soon as logs are generated from Fama's AWS environments and optimizes them into insights, statistics, and aggregates. These outputs are streamed to Datadog instead of complete datasets, while all raw data is routed to low-cost storage. If an issue or anomaly occurs, Edge Delta automatically triggers a full-fidelity log capture in Datadog. This solution gave Fama 100% visibility into their raw logs and reduced the data being indexed into Datadog by over 80%, resulting in a 60% reduction in their monthly bill. Read More Log in to view content
Contents

Technology Category

Analytics & Modeling - Machine Learning

Robots - Parallel Robots

Applicable Industries

Buildings

Construction & Infrastructure

Applicable Functions

Maintenance

Use Cases

Intrusion Detection Systems

Supply Chain Visibility

Challenge

Fama, a leading background screening company, was facing a significant challenge with its Datadog adoption. The company had recently modernized its observability stack and moved away from a self-hosted log management platform to Datadog. However, the costs associated with Datadog were much higher than projected, with Fama spending between 2-3x each month as much as the team had forecasted. This was due to two factors: Fama's growth and the size of their logs. The company had doubled its screening volume from 2020 to 2021, naturally creating more logs. Additionally, their logs were larger than anticipated, with what should have been single log events often being three or four events. Attempts to strategize ways to lower costs with Datadog either didn't make enough of a difference or created significant blindspots by filtering out logs. Fama needed a solution that would reduce the volume of data indexed into Datadog without sacrificing visibility.

About Customer

Founded in 2016, Fama is a leading background screening company that helps organizations make better hiring decisions by scraping a candidate’s public social media presence and using machine learning to identify problematic behaviors. The company supports organizations around the world, including a majority of the largest executive search firms. Fama is always looking for new, cutting-edge technologies to better support its customers. One of its core initiatives has been to streamline its infrastructure and operations by adopting a greenfield serverless architecture on AWS. The company’s upcoming releases will run solely on this serverless architecture.

Solution

Fama found its solution in Edge Delta, an observability platform that complemented and integrated with their existing Datadog implementation. Edge Delta provided out-of-the-box integrations to support Fama's serverless adoption, namely with Amazon CloudWatch and AWS Lambda. It also addressed Fama's core cost challenge without forcing them to sacrifice visibility through sampling or filtering logs. Edge Delta analyzes raw datasets as soon as logs are generated from Fama's AWS environments and optimizes them into insights, statistics, and aggregates. These outputs are streamed to Datadog instead of complete datasets, while all raw data is routed to low-cost storage. If an issue or anomaly occurs, Edge Delta automatically triggers a full-fidelity log capture in Datadog. This solution gave Fama 100% visibility into their raw logs and reduced the data being indexed into Datadog by over 80%, resulting in a 60% reduction in their monthly bill.

Impact #1	Beyond the significant cost savings, Fama also benefited from Edge Delta's automated alerting and anomaly detection. Almost immediately after turning it on, Fama started seeing anomalies. Before, the engineering team had to know what patterns to look for in order to detect and alert against them with Datadog. Now, that's all handled without any manual intervention, simply by ingesting the logs through Edge Delta before the outputs hit Datadog. This has provided Fama with a view into issues they didn't know were going on and detected anomalies on its own without them having to specify what it should be looking for. The onboarding process with Edge Delta was also smooth and quick, with Fama up and running within days. The support team was praised for being fantastic, simple, open, and responsive.

Impact #1

Beyond the significant cost savings, Fama also benefited from Edge Delta's automated alerting and anomaly detection. Almost immediately after turning it on, Fama started seeing anomalies. Before, the engineering team had to know what patterns to look for in order to detect and alert against them with Datadog. Now, that's all handled without any manual intervention, simply by ingesting the logs through Edge Delta before the outputs hit Datadog. This has provided Fama with a view into issues they didn't know were going on and detected anomalies on its own without them having to specify what it should be looking for. The onboarding process with Edge Delta was also smooth and quick, with Fama up and running within days. The support team was praised for being fantastic, simple, open, and responsive.

Benefit #1	Fama optimized the data being indexed into Datadog by over 80%.
Benefit #2	Fama cut their monthly Datadog bill by 60%.
Benefit #3	Fama achieved 100% visibility into their raw logs.

Benefit #1

Fama optimized the data being indexed into Datadog by over 80%.

Benefit #2

Fama cut their monthly Datadog bill by 60%.

Benefit #3

Fama achieved 100% visibility into their raw logs.

Download PDF Version

Overview

Fama's Journey to 100% Datadog Visibility and Anomaly Detection Automation with Edge Delta

Operational Impact

Quantitative Benefit