ClickHouse Case Studies Opensee: Harnessing Financial Big Data with ClickHouse

Edit This Case Study Record

	Opensee: Harnessing Financial Big Data with ClickHouse ClickHouse

Opensee: Harnessing Financial Big Data with ClickHouse

ClickHouse

Technology Category	Analytics & Modeling - Big Data Analytics Application Infrastructure & Middleware - Database Management & Storage
Applicable Industries	Cement Construction & Infrastructure
Applicable Functions	Procurement
Use Cases	Real-Time Location System (RTLS) Track & Trace of Assets
Services	Hardware Design & Engineering Services System Integration
Challenge	Opensee, a financial technology company, was founded by a team of financial industry and technology experts who were frustrated by the lack of simple big data analytics solutions that could efficiently handle their vast amounts of data. Financial institutions have always stored large amounts of data for decision-making processes and regulatory reasons. However, since the financial crisis, regulators worldwide have significantly increased reporting requirements, insisting on longer historical ranges and deeper granularity. This has led to an exponential increase in data, forcing financial institutions to review and upgrade their infrastructure. Unfortunately, many of the storage solutions, such as data lakes built on a Hadoop stack, were too slow for at-scale analytics. Other solutions like in-memory computing solutions and query accelerators presented issues with scalability, high hardware costs, and loss of granularity. Financial institutions were thus forced into a series of compromises. Read More
About Customer	Opensee is a financial technology company that provides real-time self-service analytics solutions to financial institutions. These solutions help financial institutions turn their big data challenges into a competitive advantage by unlocking vital opportunities led by business users. Opensee, formerly known as ICA, was started by a team of financial industry and technology experts. The company is headquartered in Paris, with offices in London and New York, and works with a trusted client base across global Tier 1 banks, asset managers, hedge funds, and trading platforms. Read More
Solution	Opensee built a solution that leverages ClickHouse, a data storage system capable of scaling horizontally with data and providing fast OLAP query response time. ClickHouse was chosen after a thorough evaluation in 2016. Opensee's platform can handle the huge volume that data lakes require and the fast response that in-memory databases can give, without the need to pre-aggregate the data. Opensee provides a series of APIs that allow users to abstract all the complexity and the physical data model. These APIs are used for data ingestion, data query, model management, etc. Opensee's back end, which provides indirect access to ClickHouse, is written in Scala, while PostgreSQL contains all the configuration and context data that must be managed transactionally. Opensee also provides various options for front ends to interact with the data, navigate through the cube, and leverage functionality like data versioning. Read More Log in to view content
Contents

Technology Category

Analytics & Modeling - Big Data Analytics

Application Infrastructure & Middleware - Database Management & Storage

Applicable Industries

Cement

Construction & Infrastructure

Applicable Functions

Procurement

Use Cases

Real-Time Location System (RTLS)

Track & Trace of Assets

Services

Hardware Design & Engineering Services

System Integration

Challenge

Opensee, a financial technology company, was founded by a team of financial industry and technology experts who were frustrated by the lack of simple big data analytics solutions that could efficiently handle their vast amounts of data. Financial institutions have always stored large amounts of data for decision-making processes and regulatory reasons. However, since the financial crisis, regulators worldwide have significantly increased reporting requirements, insisting on longer historical ranges and deeper granularity. This has led to an exponential increase in data, forcing financial institutions to review and upgrade their infrastructure. Unfortunately, many of the storage solutions, such as data lakes built on a Hadoop stack, were too slow for at-scale analytics. Other solutions like in-memory computing solutions and query accelerators presented issues with scalability, high hardware costs, and loss of granularity. Financial institutions were thus forced into a series of compromises.

About Customer

Opensee is a financial technology company that provides real-time self-service analytics solutions to financial institutions. These solutions help financial institutions turn their big data challenges into a competitive advantage by unlocking vital opportunities led by business users. Opensee, formerly known as ICA, was started by a team of financial industry and technology experts. The company is headquartered in Paris, with offices in London and New York, and works with a trusted client base across global Tier 1 banks, asset managers, hedge funds, and trading platforms.

Solution

Opensee built a solution that leverages ClickHouse, a data storage system capable of scaling horizontally with data and providing fast OLAP query response time. ClickHouse was chosen after a thorough evaluation in 2016. Opensee's platform can handle the huge volume that data lakes require and the fast response that in-memory databases can give, without the need to pre-aggregate the data. Opensee provides a series of APIs that allow users to abstract all the complexity and the physical data model. These APIs are used for data ingestion, data query, model management, etc. Opensee's back end, which provides indirect access to ClickHouse, is written in Scala, while PostgreSQL contains all the configuration and context data that must be managed transactionally. Opensee also provides various options for front ends to interact with the data, navigate through the cube, and leverage functionality like data versioning.

Impact #1	Opensee's solution, powered by ClickHouse, has provided financial institutions with a turn-key solution to analyze all their data sets, which used to be siloed, in one single place, one single data model, one single infrastructure, and all of that in real time, combining very granular and very long historical ranges. This has alleviated critical limitations of their existing solutions, avoiding legacy compromises and a lack of flexibility. The solution also includes a 'What If' simulation feature that allows business users to correct inaccurate data or simulate new values on the fly, with full auditability and traceability, without deleting any data. Furthermore, a Python processor is available to define more complex calculations, and a UI dedicated to financial institutions has been developed with and for its users.

Impact #1

Opensee's solution, powered by ClickHouse, has provided financial institutions with a turn-key solution to analyze all their data sets, which used to be siloed, in one single place, one single data model, one single infrastructure, and all of that in real time, combining very granular and very long historical ranges. This has alleviated critical limitations of their existing solutions, avoiding legacy compromises and a lack of flexibility. The solution also includes a 'What If' simulation feature that allows business users to correct inaccurate data or simulate new values on the fly, with full auditability and traceability, without deleting any data. Furthermore, a Python processor is available to define more complex calculations, and a UI dedicated to financial institutions has been developed with and for its users.

Benefit #1	Opensee's solution has led to a significant reduction in hardware costs, dividing them by ten or more.
Benefit #2	The solution allows for the use of very large datasets on standard servers on-premise or in the cloud.
Benefit #3	Opensee's solution can handle trillions of data points, allowing financial institutions to navigate very large data cubes.

Benefit #1

Opensee's solution has led to a significant reduction in hardware costs, dividing them by ten or more.

Benefit #2

The solution allows for the use of very large datasets on standard servers on-premise or in the cloud.

Benefit #3

Opensee's solution can handle trillions of data points, allowing financial institutions to navigate very large data cubes.

Download PDF Version

Overview

Opensee: Harnessing Financial Big Data with ClickHouse

Operational Impact

Quantitative Benefit