ClickHouse Case Studies Contentsquare's Successful Migration from Elasticsearch to ClickHouse: A Case Study

Edit This Case Study Record

	Contentsquare's Successful Migration from Elasticsearch to ClickHouse: A Case Study ClickHouse

Contentsquare's Successful Migration from Elasticsearch to ClickHouse: A Case Study

ClickHouse

Technology Category	Cybersecurity & Privacy - Endpoint Security Cybersecurity & Privacy - Identity & Authentication Management
Applicable Industries	Buildings Cement
Applicable Functions	Procurement Quality Assurance
Use Cases	Building Automation & Control Time Sensitive Networking
Services	System Integration Testing & Certification
Challenge	Contentsquare, a SaaS company, was facing significant challenges with its existing Elasticsearch setup. The company had 14 Elasticsearch clusters in production, each with 30 nodes. However, they were struggling with horizontal scalability, as they were unable to assemble larger clusters and maintain their stability for their workload. This limitation in cluster size meant that they could not handle any tenant that would not fit into a single cluster, severely restricting their ability to grow. The upper bound on the amount of traffic they could handle was slowing down the company's growth for technical reasons, which was unacceptable. They were left with two options: either find a way to host each tenant efficiently in a multi-cluster setup or migrate to a more scalable technology. Read More
About Customer	Contentsquare is a SaaS company that provides digital experience analytics. The company helps businesses understand how and why users are interacting with their app, mobile, and web sites. They compute billions of touch and mouse movements and transform this knowledge into profitable actions that increase engagement, reduce operational costs, and maximize conversion rates. Using behavioral data, artificial intelligence, and big data to provide automatic recommendations, Contentsquare empowers every member of the digital team to easily measure the impact of their actions and make fast and productive data-driven decisions to optimize the customer journey. Read More
Solution	Contentsquare decided to migrate to a more scalable technology and began looking into OLAP database engines that would meet their requirements of minimal latency for queries, a rich query language, efficiency with spinning disks, and simplicity in deployment and operation. After extensive engineering studies, they found that ClickHouse met all their requirements and began planning the migration. The migration process was divided into three phases: getting familiar with ClickHouse and building a new product with it, mirroring all the existing features with custom tooling to ensure no regression, and migrating their clients one by one. They built a new product on top of ClickHouse to familiarize themselves with the technology, then migrated their main product iteratively, rewriting each API endpoint one by one to use ClickHouse instead of Elasticsearch. Finally, they migrated their customers to the new infrastructure, taking care not to move everyone at once to identify potential issues. Read More Log in to view content
Contents

Technology Category

Cybersecurity & Privacy - Endpoint Security

Cybersecurity & Privacy - Identity & Authentication Management

Applicable Industries

Buildings

Cement

Applicable Functions

Procurement

Quality Assurance

Use Cases

Building Automation & Control

Time Sensitive Networking

Services

System Integration

Testing & Certification

Challenge

Contentsquare, a SaaS company, was facing significant challenges with its existing Elasticsearch setup. The company had 14 Elasticsearch clusters in production, each with 30 nodes. However, they were struggling with horizontal scalability, as they were unable to assemble larger clusters and maintain their stability for their workload. This limitation in cluster size meant that they could not handle any tenant that would not fit into a single cluster, severely restricting their ability to grow. The upper bound on the amount of traffic they could handle was slowing down the company's growth for technical reasons, which was unacceptable. They were left with two options: either find a way to host each tenant efficiently in a multi-cluster setup or migrate to a more scalable technology.

About Customer

Contentsquare is a SaaS company that provides digital experience analytics. The company helps businesses understand how and why users are interacting with their app, mobile, and web sites. They compute billions of touch and mouse movements and transform this knowledge into profitable actions that increase engagement, reduce operational costs, and maximize conversion rates. Using behavioral data, artificial intelligence, and big data to provide automatic recommendations, Contentsquare empowers every member of the digital team to easily measure the impact of their actions and make fast and productive data-driven decisions to optimize the customer journey.

Solution

Contentsquare decided to migrate to a more scalable technology and began looking into OLAP database engines that would meet their requirements of minimal latency for queries, a rich query language, efficiency with spinning disks, and simplicity in deployment and operation. After extensive engineering studies, they found that ClickHouse met all their requirements and began planning the migration. The migration process was divided into three phases: getting familiar with ClickHouse and building a new product with it, mirroring all the existing features with custom tooling to ensure no regression, and migrating their clients one by one. They built a new product on top of ClickHouse to familiarize themselves with the technology, then migrated their main product iteratively, rewriting each API endpoint one by one to use ClickHouse instead of Elasticsearch. Finally, they migrated their customers to the new infrastructure, taking care not to move everyone at once to identify potential issues.

Impact #1	The migration to ClickHouse has brought significant operational benefits to Contentsquare. The company has been able to overcome the limitations of its previous Elasticsearch setup, enabling it to handle larger volumes of data and traffic, and thus support its growth. The migration process, though complex, was carried out without any regression, thanks to careful planning and a phased approach. The company has also been able to leverage the capabilities of ClickHouse to optimize its query performance, resulting in faster and more efficient data processing. This has not only improved the performance of Contentsquare's applications but also enhanced the experience for its customers, who can now access more historical data and benefit from improved data retention.

Impact #1

The migration to ClickHouse has brought significant operational benefits to Contentsquare. The company has been able to overcome the limitations of its previous Elasticsearch setup, enabling it to handle larger volumes of data and traffic, and thus support its growth. The migration process, though complex, was carried out without any regression, thanks to careful planning and a phased approach. The company has also been able to leverage the capabilities of ClickHouse to optimize its query performance, resulting in faster and more efficient data processing. This has not only improved the performance of Contentsquare's applications but also enhanced the experience for its customers, who can now access more historical data and benefit from improved data retention.

Benefit #1	ClickHouse turned out to be 11 times cheaper in infrastructure cost.
Benefit #2	There was a 10x performance improvement in their p99 for queries.
Benefit #3	Customers can now query up to 3 months of historical data instead of 1.

Benefit #1

ClickHouse turned out to be 11 times cheaper in infrastructure cost.

Benefit #2

There was a 10x performance improvement in their p99 for queries.

Benefit #3

Customers can now query up to 3 months of historical data instead of 1.

Download PDF Version

Overview

Contentsquare's Successful Migration from Elasticsearch to ClickHouse: A Case Study

Operational Impact

Quantitative Benefit