ClickHouse Case Studies Highlight.io's Observability Solution Powered by ClickHouse: A Comprehensive Case Study

Edit This Case Study Record

	Highlight.io's Observability Solution Powered by ClickHouse: A Comprehensive Case Study ClickHouse

Highlight.io's Observability Solution Powered by ClickHouse: A Comprehensive Case Study

ClickHouse

Technology Category	Application Infrastructure & Middleware - Middleware, SDKs & Libraries Platform as a Service (PaaS) - Application Development Platforms
Applicable Industries	Cement Construction & Infrastructure
Applicable Functions	Product Research & Development
Use Cases	Behavior & Emotion Tracking Leakage & Flood Monitoring
Services	Cloud Planning, Design & Implementation Services System Integration
Challenge	Highlight.io, an open-source observability platform, initially focused on session replay and frontend web development features. However, as the need for full-stack observability grew, the platform needed to expand its offerings. This expansion was necessary to enable developers to track user experiences within web apps, identify backend errors, and analyze associated logs across their infrastructure. The challenge was to integrate these features into a single-pane view to streamline the troubleshooting process. Furthermore, the platform aimed to add logging capabilities to its stack, powered by ClickHouse, to provide deeper insights into applications by capturing and analyzing server-side logs. The goal was to handle high data ingestion rates and ensure that developers could access up-to-date information in real-time. Read More
About Customer	Highlight.io's customers are developers who need a comprehensive observability solution for their applications. These developers require a platform that allows them to track user experiences within web apps, identify backend errors, and analyze associated logs across their infrastructure. They need a solution that can handle high data ingestion rates and provide real-time information. The customers also require a platform that is easy to install and integrates seamlessly with their existing systems. They value the ability to quickly access and analyze user sessions, logs, and error information to identify and resolve issues more effectively. Furthermore, they appreciate a platform that can handle multi-tenancy and scaling. Read More
Solution	Highlight.io integrated ClickHouse into its platform to handle high data ingestion rates and provide real-time information. The platform's architecture, which includes a front-end built with React and TypeScript, and a back-end built using Go, leverages ClickHouse for storing log data and combining it with information from other data stores. This integration allows developers to quickly access and analyze user sessions, logs, and error information, helping them identify and resolve issues more effectively. To install Highlight.io, customers simply add a few lines of code by installing their respective Highlight.io SDK. The platform uses a cloud-hosted OpenTelemetry agent, publicly exposed for customers’ SDKs to connect to, which wraps OpenTelemetry SDKs to keep installation simple. Upon ingestion, data is stored in a single table with a project ID as part of the primary key for handling multi-tenancy and scaling. Read More Log in to view content
Contents

Technology Category

Application Infrastructure & Middleware - Middleware, SDKs & Libraries

Platform as a Service (PaaS) - Application Development Platforms

Applicable Industries

Cement

Construction & Infrastructure

Applicable Functions

Product Research & Development

Use Cases

Behavior & Emotion Tracking

Leakage & Flood Monitoring

Services

Cloud Planning, Design & Implementation Services

System Integration

Challenge

Highlight.io, an open-source observability platform, initially focused on session replay and frontend web development features. However, as the need for full-stack observability grew, the platform needed to expand its offerings. This expansion was necessary to enable developers to track user experiences within web apps, identify backend errors, and analyze associated logs across their infrastructure. The challenge was to integrate these features into a single-pane view to streamline the troubleshooting process. Furthermore, the platform aimed to add logging capabilities to its stack, powered by ClickHouse, to provide deeper insights into applications by capturing and analyzing server-side logs. The goal was to handle high data ingestion rates and ensure that developers could access up-to-date information in real-time.

About Customer

Highlight.io's customers are developers who need a comprehensive observability solution for their applications. These developers require a platform that allows them to track user experiences within web apps, identify backend errors, and analyze associated logs across their infrastructure. They need a solution that can handle high data ingestion rates and provide real-time information. The customers also require a platform that is easy to install and integrates seamlessly with their existing systems. They value the ability to quickly access and analyze user sessions, logs, and error information to identify and resolve issues more effectively. Furthermore, they appreciate a platform that can handle multi-tenancy and scaling.

Solution

Highlight.io integrated ClickHouse into its platform to handle high data ingestion rates and provide real-time information. The platform's architecture, which includes a front-end built with React and TypeScript, and a back-end built using Go, leverages ClickHouse for storing log data and combining it with information from other data stores. This integration allows developers to quickly access and analyze user sessions, logs, and error information, helping them identify and resolve issues more effectively. To install Highlight.io, customers simply add a few lines of code by installing their respective Highlight.io SDK. The platform uses a cloud-hosted OpenTelemetry agent, publicly exposed for customers’ SDKs to connect to, which wraps OpenTelemetry SDKs to keep installation simple. Upon ingestion, data is stored in a single table with a project ID as part of the primary key for handling multi-tenancy and scaling.

Impact #1	The integration of ClickHouse into Highlight.io's platform has resulted in a robust and scalable observability solution. Developers can now monitor their applications more effectively, ensuring optimal performance. The single-pane view streamlines the troubleshooting process, saving developers time and effort. The addition of logging capabilities provides deeper insights into applications, enabling developers to identify and resolve issues more effectively. The platform's easy installation process and seamless integration with existing systems have also improved user experience. Furthermore, Highlight.io's commitment to open-source development and continuous product enhancement has resulted in a platform that is constantly evolving to meet the needs of developers.

Impact #1

The integration of ClickHouse into Highlight.io's platform has resulted in a robust and scalable observability solution. Developers can now monitor their applications more effectively, ensuring optimal performance. The single-pane view streamlines the troubleshooting process, saving developers time and effort. The addition of logging capabilities provides deeper insights into applications, enabling developers to identify and resolve issues more effectively. The platform's easy installation process and seamless integration with existing systems have also improved user experience. Furthermore, Highlight.io's commitment to open-source development and continuous product enhancement has resulted in a platform that is constantly evolving to meet the needs of developers.

Benefit #1	High data ingestion rates enabled by ClickHouse integration
Benefit #2	Real-time access to up-to-date information
Benefit #3	Quick access and analysis of user sessions, logs, and error information

Benefit #1

High data ingestion rates enabled by ClickHouse integration

Benefit #2

Real-time access to up-to-date information

Benefit #3

Quick access and analysis of user sessions, logs, and error information

Download PDF Version

Overview

Highlight.io's Observability Solution Powered by ClickHouse: A Comprehensive Case Study

Operational Impact

Quantitative Benefit