X

As the top online retailer in the Netherlands and Belgium, bol.com has grown massively in a short amount of time. As the company scaled, the data began evaluating alternatives to their overloaded Hadoop cluster that was taking too long to run some jobs. At the time, the company’s analysts were using Platfora for data preparation and visualization. Shortly after the go-live, Platfora announced its acquisition by Workday and with that the discontinuation of the product. With this as a catalyst, bol.com began looking for a new solution to support their BI and analytics program. Self-service was a top priority for the bol.com team. As they looked for new technology partners, they wanted to integrate a semantic layer solution that could cover all data assets, now and in the future. Further, they wanted to ensure compatibility with whatever BI and analysis tools they may use in the future.

Rakuten, a shopping rewards company, had moved from their initial SQL database in 2014 to an AtScale-powered Hadoop solution in 2018. However, this wasn’t sufficient and they soon began to experience a resource crunch based on the sheer size of their database. Rakuten's existing architecture meant that business users didn't have the computing resources necessary to work with large datasets. This led to competition between business units for hard disk access, memory, and CPU time. The internal team was frustrated with the competition for resources, and the operational overhead and associated hardware and electricity costs also meant the solution was no longer cost-efficient. That, coupled with the continuous processing demands on storage infrastructure, forced Rakuten to consider new solutions for their data needs. They knew they needed more processing capability and flexibility to continue serving their customers effectively.

Affinity Federal Credit Union (AFCU), a large member-owned credit union, was looking for opportunities to better leverage their data assets to improve service to their more than 185,000 members. They had been relying on legacy analytics infrastructure tools like ModelMax or Dundas BI, which required too much manual effort and slowed down decision-making. AFCU had been partnered with a Credit Union Service Organization (CUSO) that provided analytics-as-a-service, but this approach was slow and uncontrollable, often getting in the way of decision making and making it difficult to grow internal understanding of data. AFCU realized they couldn’t remain reliant on an outsourced analytics team and legacy processes to unearth insights from their data. It was time to transition to a modern, self-service BI program to allow faster, data-backed decision-making at scale.

A Fortune 50 retailer launched an initiative to modernize their analytics infrastructure with the primary goal of increasing the flow of data-driven insights that could lead to improved margins, optimization of product mix and better inventory management. Their challenge was to enable better analytics at scale while ensuring efficiency and consistency across a broad audience of data consumers. With thousands of users performing analytics using a diverse set of legacy platforms, including SQL Server Analysis Services (SSAS), Teradata, and Hadoop, the existing infrastructure was expensive and could not scale at the rate of their business. To empower their users, the data team needed a scalable semantic layer solution that could serve the needs of internal users as well as suppliers that rely on a shared view of inventory. The solution needed to scale, needed to support security and access control policies, and needed to support the organization’s migration from on-premise legacy data platforms to a cloud data warehouse.

The 2012 U.S. Women’s Olympic cycling team was looking for a competitive edge after a disappointing finish in the World Championships. They turned to Olympic cyclist Sky Christopherson, who had used the quantified-self movement in his training to break a world record. Christopherson established an experimental project to help the team record and analyze relevant data that could reveal actionable insights for optimizing their athletic performance. The team faced the challenge of recording relevant data, integrating it, analyzing it, and visualizing all of these data points to reveal insights they could incorporate into training. The sheer amount of data, and with each device producing different types of data (often in unstructured formats) meant that traditional database and business intelligence technologies were not an option.

The Detroit Crime Commission (DCC) was created to combat the high crime rates in Detroit, which have been exacerbated by severe budget cuts to law enforcement agencies. The DCC's main effort has been to identify individuals known to be engaged in dangerous criminal activities. They compiled terabytes of proprietary and public crime-related data related to these individuals' activities. However, they needed a way to quickly and easily aggregate and analyze this data to identify and prevent ongoing or planned criminal activity. The DCC had tried other data collection and analysis tools but found them lacking. These tools were only good at collecting a sample of the data, and they did not perform the relevant analyses needed by the Commission staff.

Vivint, the largest home automation company in North America, was facing a challenge with their analytics infrastructure platform, Hadoop. The team was spending too much time on mundane, technical tasks preparing and integrating the data rather than doing actual, value-added analysis. They were looking for a solution that could make their staff more efficient and seamlessly integrate with their Hadoop analytics platform. Another key consideration was the ability to integrate and analyze not just row data but also streaming data, which is a key component to their smart home analytics solution to big data for Internet of Things.

Sophos, a company that has been producing antivirus and encryption products for nearly 30 years, was facing a challenge with the increasing complexity of IT networks and the sophistication of threats and attacks. The company's products examine billions of events per day to detect malicious files, with over 300,000 new potentially malicious files reported to SophosLabs daily for analysis. The volume and complexity of the data grew to a point where their old analytic infrastructure could not keep pace. Another challenge was the cloud telemetry data consisting of billions of lookups for website and file information. A particular aspect of the analysis – correlating patterns across previous analysis – had become too complex for their SQL-based database and analytic tools to manage. Sophos investigated NoSQL technologies available at the time and selected Hadoop for big data analytics needs related to telemetry and threat correlation. However, out-of-the-box Hadoop was lacking any enterprise-ready tools for creating analytic reports, dashboards, data access controls or mechanisms to easily import or export data in and out of various storage systems.

Yapı Kredi, the fourth largest private bank in Turkey, wanted to become a more data-driven company to increase business agility, reduce operating expenses, and improve the overall customer experience. However, they faced the challenge of deriving value from their vast amount of data, most of which was structured and stored in a traditional relational data warehouse. Their traditional business intelligence tools were too inflexible and forced a waterfall approach, which was time and resource-consuming. The rigid data schemas required before moving to the analysis step every time made the process laborious and slow. Yapı Kredi needed a more agile toolset for the iterative process of data discovery that’s important for any analysis.

Cancer diagnosis is complicated due to the uniqueness of each case, and treatment outcomes vary greatly from patient to patient. DKFZ, the largest biomedical research Institute in Germany, is working to understand the mechanisms of cancer, identify risk factors, and find new ways to prevent people from getting cancer. A key focus of DKFZ’s medical researchers is genomic data research. However, due to the massive volumes of genomic data involved in this research, DKFZ faced huge challenges on the data and analytics front. Their analytic systems were overwhelmed by many petabytes of data, and analyzing an entire patient data set took weeks and even months to complete. These huge bottlenecks greatly slowed research and frustrated staff.

SHOEPASSION.com had disparate systems running different parts of its business operations, which was hindering their aggressive expansion plans. With data in silos from their ecommerce, analytics, and ERP systems, they wanted to add in data from Google Ads, Google reports and docs, and excel spreadsheets to gain better customer and operational insights. On the marketing side, SHOEPASSION.com wanted to increase the yield in marketing activities by analyzing customer orders, revenue, and churn. It was also important to analyze their costs and ROI from various marketing channels. On the operational side, SHOEPASSION.com faced challenges in managing product inventory, distribution, and delivery. To minimize their costs, they needed to keep the minimal level of inventory to satisfy their customer demands in the various geographies.

A restaurant chain approached a Tier 1 mobile service provider to help launch a campaign sending messages to consumers’ mobile phones, encouraging people to download the restaurant’s free app. The messages were sent to large audience segments, created on the basis of standard demographic information, such as device type and age group of opt-in subscribers. However, the ad campaign was costly, as it was being sent to millions of mobile users, and was yielding poor results – about .6 app downloads per every 100 messages sent. The restaurant chain challenged the mobile operator to improve the conversion rate, agreeing to pay more for each download if the operator could better target their customers.

The marketing team of a leading Mobile Service Provider had a plan to offer advertising agencies target market insights to optimize their advertising ROI. They planned to gather subscriber analytics including demographic, geographic and behavioral statistics in an anonymous format, and offer this information to current and prospective advertising customers. However, due to the size of the network and volume of data that needed to be analyzed, the option of rolling out a deep packet inspection solution was extremely cost prohibitive and could not be done within the timeframe needed. The complexity of correlating content data with subscriber demographics and geographical location, put the marketing team’s initiative beyond reach.

The corporation, a leading Multiple-System Operator in North America, was facing challenges with delayed problem resolution which was affecting customer satisfaction. The Operations Team was unable to identify the root of the problem and rapidly distinguish between customer premise problems and headend CMTS or video server issues. This led to unnecessary dispatch of technicians to homes, which often turned out to be a headend problem instead of an individual set-top box problem. This resulted in customer frustration and a negative impact on their Net Promoter Score.

A European Multiple-System Operator (MSO) was struggling to rapidly distinguish between issues caused by customer premise devices and headend equipment. This delayed the MSO’s ability to find the root cause of problems and subsequently resolve the issues. With the cost of a truck roll in Germany running about 60 to 70 euros and the handling of incoming customer service calls running about 5 to 10 euros each, the provider hoped to reduce customer service costs and improve customer satisfaction at the same time.

The marketing team of a Mobile Network Operator (MNO) was struggling to build rich customer profiles to present relevant offers and ads to their subscribers. They needed to match individual preferences and browsing behaviors with subscriber IDs. Additionally, they needed to accurately categorize website URLs viewed with accuracy levels of 80% or greater. However, they were unable to achieve this with the software tools they had in place.

Broadcom, a global communications semiconductor powerhouse, was facing challenges in generating new analytical dashboards or reports based on data from various software solutions, including Workday, Oracle ERP, Model N, Oracle Demantra, and Microsoft Excel. The process was time-consuming, taking up to twelve weeks. As technology advanced, Broadcom sought to shorten this dashboard and report development cycle and reduce report run times. The company was looking for a solution that could integrate with other applications, especially Microsoft Excel, which is used by 90% of the company. The solution also needed to be cost-effective, easy to maintain, and minimize the number of peripheral technologies required for solution support.

As a fast-growing company with a rapidly-expanding customer base, Toast found itself inundated with data and an increasing number of data sources. With no analytics solution in place, employees used a combination of giant Excel spreadsheets and Salesforce dashboards. Everyone acted as their own Excel analyst, passing around spreadsheets, discussing how to look at the data, and debating why their numbers didn’t match up. The Salesforce dashboards employees utilized varied tremendously—even the same dashboards were viewed using different filters. Each minute spent discussing discrepancies in the data meant less time focused on the important issues facing the company. Without quick access to consistent information it became harder and harder to make timely decisions.

The Land and Buildings Department at a top ten U.S. university was facing significant delays in accessing analytics, which was reducing revenue and negatively impacting their budget. With over 300 buildings to manage, the department generates work orders within Oracle E-Business Suite (EBS) for maintenance tasks. However, the existing BI tool was slow in pulling data from EBS, causing delays in work completion. Technicians would sometimes wait up to 15 minutes for a response to basic queries, eating into their billable time.

The investment bank arm of a global financial institution was struggling to limit its daily risk exposure across its entire business. The risk analytics team found it difficult to assess their daily trading positions across various asset classes - foreign exchange, equities, fixed income, and other special products. They were unable to see how one asset class risk affected another. The bank set out to deliver a daily single consolidated view of its risk position to drill down into individual transactions. However, they faced several challenges: With over a billion risk points a day, they were struggling to create a consolidated view of risk across their assets. It was impossible to correlate risks across asset classes to understand trends. Analysts were unable to drill down into their data to understand complex transactions. Risk analysis was always late and deficient.

The leading apparel and footwear brand faced challenges in fine-tuning its material forecasting based on consumer demand patterns. With an extensive network of factories servicing global stores, it was difficult to estimate the exact quantity and type of raw materials for different manufacturing locations. The existing BI architecture did not allow them to analyze more than 18 weeks of data. They were pulling source data from Amazon S3 to Snowflake, building aggregates on Azure Analysis Services (AAS), and then performing analysis on Excel. This led to multiple points of failure and each hop had an associated cost. They were hitting the limits of AAS in terms of processing that could be done and missed SLAs due to high data volumes during the holiday season. As data volumes rose, Excel reports would often freeze/crash.

The pharmacy chain, with over 9,500 stores across the US, 20,000 suppliers, and 1 million products, generated 17 billion records of transaction-level data each day. They wanted to analyze two years of supply chain data to drive their business decisions. However, they faced several challenges in analyzing the continuously growing supply chain data. Their data was coming from a wide variety of internal and external sources and living in multiple applications such as Netezza, Oracle, Excel, and more. Despite consolidating it on an on-premise data lake, they faced several challenges in analyzing the continuously growing supply chain data. Some of the key challenges they faced included hundreds of billions of records from 50+ sources, inability to scale up to billions of rows, slow response times kept business users waiting for days to get insights, and diagnostics were difficult as they could not drill down to granular details.

The grocery chain wanted to use their data to understand their customers better and maintain their market-leader position. They decided to align their merchandise mix and store inventory to match each customer’s specific needs. With hundreds of stores, thousands of products, and daily visitors, the store had details on almost 140,000 transactions per day and wanted to use this data to support their business decisions. However, their current environment could not handle the data scale and complexity, making it impossible to conduct Year-over-Year, much less Month-over-Month analysis. They resorted to writing complex queries using Impala to fetch data from their Cloudera platform. But when they tried to join tables with more than one billion cardinalities, Impala usually timed out.

The pharmacy store chain, with over 20,000 national and international suppliers, was struggling with the efficiency of its supply chain network. The company had billions of records from over 50 tables and sources, making analysis difficult. There was no single source of truth as data was decentralized and coming from different sources. Reporting was time-consuming as teams had to compile reports manually and could send them only once a month to top 200 suppliers. There was no way to provide secure, self-service access to massive volumes of supply chain data to external users/suppliers.

The telecom company was facing several analytical challenges. The time-to-insights was too long for ad hoc analysis, they first had to write, tune, and optimize their queries and then run them to get the required reports. As massive datasets were joined at runtime, queries would take 10 to 30 minutes to return, and complex ones would take even longer. As the business grew, it became difficult to meet the aggressive timelines of its users. In addition, though their data platform allowed them to retain 15 months of data, it was very challenging to analyze across multiple months due to the volume and complexity of their data.

The global telecommunication company was facing several challenges with their existing MicroStrategy BI tool. As the volume of their data increased massively, and more dimensions, metrics, and measures got added to their MicroStrategy reports, response times increased from seconds to several minutes, even for smaller and less complex reports. Transferring billions of rows of raw data from the data lake to MicroStrategy for computations caused the application to time out in most cases. Disconnects between the data collected by different departments such as marketing, finance, product development, and others made cross-functional analysis extremely difficult. Building cubes on MicroStrategy consumed 50% of resources, resulting in prohibitive costs. They also faced inconsistent response times on MicroStrategy queries and an inability to meet increasing business requirements as they could not accommodate additional dimensions and measures in reports without impacting performance.

The e-waste recycling company was facing several analytical challenges on their Snowflake / AWS / Tableau platform. They were experiencing severe degradation in query performance while analyzing multiple years of data in Snowflake. There was no semantic layer to define consistent data models and standardize KPI calculations. The company was also dealing with unpredictable querying costs on their Snowflake data warehouse, with bills running very high at times. They found it difficult to model their data and deal with multi-level hierarchy and one-to-many joins between facts.

A leading multinational investment bank and financial services company operating in 50 countries wanted to establish an executive-level global view of its financial data to support Management Discussion & Analysis (MD&A) for their C-suite. Their executives needed a single dashboard where they could measure the company’s performance on a comprehensive set of real-time measures and drill down to the lowest levels of granularity instantly. They also wanted to enable quick ad hoc analysis for 700 analysts located across the globe. Their reporting requirements were quite complex, and they needed a single view across all international businesses. It was difficult to use their existing BI environment to model complex financial KPIs and build reports on massive financial records. With no tolerance for latency, it became challenging to meet the high expectations and aggressive timelines from the executive team.

The US-based financial technology services company was struggling with creating and automating a single consolidated view of all its data for hundreds of users. Despite having uniform data, reporting was siloed due to the inherent limitations of the analytical environment. They initially attempted to build MicroStrategy intelligent cubes from data residing in Google BigQuery. However, there were limits on the amount of data each cube could hold. As a result, they were forced to split their data and create separate cubes for combinations of geographic regions and data sources. This led to an inability to get a single consolidated view of their data as they were using 20 different cubes for core reporting. The time to publish the cube, even for a single data source, exceeded 6 hours. Changes in data or incremental refreshes required full reprocess downtime. Reprocessing took over 24 hours, and users could not access the system for this duration. They maxed out the capacity of their Google environment. Despite running on a large server, there was no room for additional data. They were spending massive amounts on cloud computing.

The multinational computer software company wanted to consolidate the data from different customer touchpoints and create a 360-degree view of the customer interactions with its products and services. They faced several challenges in analyzing the enormous amount of data being generated. A wide variety of customer data from sources such as call centers, web interactions, customer churn, marketing, purchase, and product usage made it difficult to get a consolidated view. Disconnects between the data collected by different departments made cross-functional analysis difficult. Non-standardized reporting between business units, with over 1000 analysts reporting on 80 customer metrics collected from more than 20 source systems. Different business units used different BI tools and were reluctant to adopt new tools.