Aravo Solutions Case Studies Fueling Growth with Predictive Models and Improved Customer Experience: A Case Study on Explorium

Edit This Case Study Record

	Fueling Growth with Predictive Models and Improved Customer Experience: A Case Study on Explorium Aravo Solutions

Fueling Growth with Predictive Models and Improved Customer Experience: A Case Study on Explorium

Aravo Solutions

Technology Category	Platform as a Service (PaaS) - Application Development Platforms Robots - Parallel Robots
Applicable Industries	Buildings Cement
Applicable Functions	Quality Assurance Warehouse & Inventory Management
Use Cases	Building Automation & Control Time Sensitive Networking
Services	Testing & Certification
Challenge	Explorium, a company that integrates organizations' data with the world's most reliable sources for predictive modeling and informed business decisions, was facing a challenge. The company was seeking to minimize data latency and free its data engineers from the task of building ELT pipelines. Explorium's platform determines the characteristics of the data and identifies potential enrichments it can make. However, the company was struggling with loading the right data quickly, regardless of the technical challenges it faced on the back end. The company was using Amazon EMR to run its ELT pipelines but realized its data engineers were spending too much time building these pipelines. This was slowing down the release of new data products and the onboarding of new data sets to its platform. Read More
About Customer	Explorium is a technology company that enables organizations to find the right data, build predictive models, and make informed business decisions. It integrates its customers' data with the world's most reliable sources, creating a powerful platform built on top of these data sources. The company combines these two assets into a valuable product. Explorium’s customers rely on the platform to enrich their existing business data according to their specific needs. The company's platform determines the characteristics of the data and identifies the potential enrichments it can make. Customers who upload massive data sets can see their results in a few hours, while customers who upload smaller data sets can see immediate enrichment. Read More
Solution	To overcome these challenges, Explorium implemented Databricks Lakehouse Platform and dbt. Databricks offered auto-scaling features and sophisticated libraries for Delta tables, which freed the company’s engineers from optimizing tables and checking file sizes. These tasks are automated in Databricks, which saved time for engineers and allowed them to concentrate on building infrastructure. Explorium now adheres to the medallion architecture, which describes three data layers of different quality that are to be stored in the lakehouse. To load raw data into the bronze layer, Explorium uses Databricks Auto Loader. To load validated data into the silver layer, the company built a transformation in SQL. For enriched data that belongs in the gold layer, Explorium extracts data from Delta Lake tables and ingests it into a warehouse or database. The Explorium platform will then retrieve data directly from this database or warehouse and serve it up to customers. dbt provides testing capabilities while eliminating the need to get data engineers involved to help with cluster definitions and sizes, permissions to connect to AWS resources, and other complex needs. Read More Log in to view content
Contents

Technology Category

Platform as a Service (PaaS) - Application Development Platforms

Robots - Parallel Robots

Applicable Industries

Buildings

Cement

Applicable Functions

Quality Assurance

Warehouse & Inventory Management

Use Cases

Building Automation & Control

Time Sensitive Networking

Services

Testing & Certification

Challenge

Explorium, a company that integrates organizations' data with the world's most reliable sources for predictive modeling and informed business decisions, was facing a challenge. The company was seeking to minimize data latency and free its data engineers from the task of building ELT pipelines. Explorium's platform determines the characteristics of the data and identifies potential enrichments it can make. However, the company was struggling with loading the right data quickly, regardless of the technical challenges it faced on the back end. The company was using Amazon EMR to run its ELT pipelines but realized its data engineers were spending too much time building these pipelines. This was slowing down the release of new data products and the onboarding of new data sets to its platform.

About Customer

Explorium is a technology company that enables organizations to find the right data, build predictive models, and make informed business decisions. It integrates its customers' data with the world's most reliable sources, creating a powerful platform built on top of these data sources. The company combines these two assets into a valuable product. Explorium’s customers rely on the platform to enrich their existing business data according to their specific needs. The company's platform determines the characteristics of the data and identifies the potential enrichments it can make. Customers who upload massive data sets can see their results in a few hours, while customers who upload smaller data sets can see immediate enrichment.

Solution

To overcome these challenges, Explorium implemented Databricks Lakehouse Platform and dbt. Databricks offered auto-scaling features and sophisticated libraries for Delta tables, which freed the company’s engineers from optimizing tables and checking file sizes. These tasks are automated in Databricks, which saved time for engineers and allowed them to concentrate on building infrastructure. Explorium now adheres to the medallion architecture, which describes three data layers of different quality that are to be stored in the lakehouse. To load raw data into the bronze layer, Explorium uses Databricks Auto Loader. To load validated data into the silver layer, the company built a transformation in SQL. For enriched data that belongs in the gold layer, Explorium extracts data from Delta Lake tables and ingests it into a warehouse or database. The Explorium platform will then retrieve data directly from this database or warehouse and serve it up to customers. dbt provides testing capabilities while eliminating the need to get data engineers involved to help with cluster definitions and sizes, permissions to connect to AWS resources, and other complex needs.

Impact #1	The implementation of Databricks Lakehouse Platform and dbt has significantly improved Explorium's operations. The company's engineers are now free from optimizing tables and checking file sizes, allowing them to focus on building infrastructure. The use of Databricks Auto Loader and SQL transformations has streamlined the data loading process, ensuring the right data is loaded quickly. The use of dbt has also ensured thorough testing of pipelines before deployment, eliminating the need for data engineers to assist with complex needs. This has resulted in a faster release of new data products and an increase in the number of data sets brought online. Overall, the new system has led to a better experience for Explorium’s customers.

Impact #1

The implementation of Databricks Lakehouse Platform and dbt has significantly improved Explorium's operations. The company's engineers are now free from optimizing tables and checking file sizes, allowing them to focus on building infrastructure. The use of Databricks Auto Loader and SQL transformations has streamlined the data loading process, ensuring the right data is loaded quickly. The use of dbt has also ensured thorough testing of pipelines before deployment, eliminating the need for data engineers to assist with complex needs. This has resulted in a faster release of new data products and an increase in the number of data sets brought online. Overall, the new system has led to a better experience for Explorium’s customers.

Benefit #1	Faster release of new data products to the platform by 10x
Benefit #2	Number of data sets brought online quarter-over-quarter increased by 2x
Benefit #3	Jobs run more quickly on Databricks than they did on EMR

Benefit #1

Faster release of new data products to the platform by 10x

Benefit #2

Number of data sets brought online quarter-over-quarter increased by 2x

Benefit #3

Jobs run more quickly on Databricks than they did on EMR

Download PDF Version

Overview

Fueling Growth with Predictive Models and Improved Customer Experience: A Case Study on Explorium

Operational Impact

Quantitative Benefit