Snorkel AI Case Studies Big Four Consulting Firm Leverages NLP for Efficient Auditing with Snorkel Flow
Edit This Case Study Record
Snorkel AI Logo

Big Four Consulting Firm Leverages NLP for Efficient Auditing with Snorkel Flow

Snorkel AI
Analytics & Modeling - Machine Learning
Sensors - Liquid Detection Sensors
Education
Finance & Insurance
Product Research & Development
Quality Assurance
Chatbots
Machine Translation
Data Science Services
Training
A globally renowned consulting firm, with a history spanning over a century, was seeking to enhance its auditing capabilities by leveraging artificial intelligence. The firm's reputation hinged on its ability to conduct thorough audits, irrespective of their size, complexity, or location. The firm's experts were spending significant time manually reviewing various accounting, auditing, and industry information, a process that was both time-consuming and costly. The firm estimated that each auditor search lasted 10 minutes and cost $50-60 on average. The firm's data science team was tasked with streamlining news monitoring to anticipate changes in capital markets, regulatory trends, or technological innovation. They aimed to use custom NLP models to automatically analyze, categorize, and extract key client information from various sources. However, they faced challenges in labeling training data for the machine learning algorithms. It took three experts a week to label 500 training data points, and they found it nearly impossible to adapt to changes in data or business goals on the fly.
Read More
The customer is a global consulting firm, part of the 'big four', with a history of over 170 years. The firm has built its reputation on rigorous analysis and the ability to anticipate change. It strives to provide its diverse team of experts with the most current and relevant accounting, auditing, and industry information. The firm has over 300,000 experts spread across 150 countries who spend hours each day manually reviewing various accounting, auditing, and industry information. The firm's data science team was tasked with leveraging AI/ML to reduce this costly effort and to innovate by helping to identify which of their clients are most likely to be audited.
Read More
The firm partnered with Snorkel AI to build an NLP application that could classify whether news articles were relevant to the original search term. Snorkel Flow's programmatic labeling capabilities allowed the team to label over 10,000 news articles in just a few hours, a task that would have taken a human audit team nearly a year. The data labeled using Snorkel Flow was 32% more accurate than their previous hand-labeling methods. The team also used Snorkel Flow's guided error analysis to rapidly improve model quality and drive fine-grained corrections, bringing their F1 score up from 70 to 85 in just a matter of days. Snorkel Flow also allowed the team to quickly react to changing market conditions and business needs by adjusting labeling functions rather than relabeling manually. The collaboration between the SMEs and data scientists was significantly improved using Snorkel Flow's Annotator Suite.
Read More
The implementation of Snorkel Flow brought about significant operational improvements for the consulting firm. The platform's programmatic labeling capabilities allowed the team to label a large volume of data in a short period, significantly reducing the time and cost associated with manual labeling. The data labeled using Snorkel Flow was more accurate, leading to improved model quality. The platform's guided error analysis enabled the team to rapidly improve model quality and make fine-grained corrections. The firm was also able to quickly adapt to changing market conditions and business needs by adjusting labeling functions, rather than having to relabel manually. The collaboration between SMEs and data scientists was greatly improved, making better use of auditors' expertise while reducing the time required of these SMEs by one-third.
10,000 news articles were auto-labeled in a few days instead of months.
Model accuracy increased by 15% over hand labeling.
Development was 3 times faster while requiring 1/3 less SME time.
Download PDF Version
test test