Getting to ROI with AI in the enterprise
The hype around artificial intelligence today is exciting but it also creates confusion, misinformation and skepticism in the minds of both data scientists and their business colleagues. Despite its promise, and its growing adoption, there is still too little we can point to in terms of real business results.
AI has been around for over 60 years, primarily in the form of Machine Learning (think structured regression models built in spreadsheets for stock trading or social science studies) but limited in its use. The recent interest in the broader application of AI can be attributed to two fundamental facts.
- The advancement in, and wide availability of powerful algorithms, specifically Deep Learning (neural networks) - making AI more relevant to a broader set of enterprise applications.
- Ubiquitous GPU (graphics processing unit) computing. Originally used for video game applications, modern GPUs can now handle more general and sophisticated computation tasks. A $500 GPU is now as powerful for running AI models as a hundred-million dollar supercomputer 1-2 decades ago.
These advances create enormous opportunity for AI in the enterprise. This article looks at some key considerations for effectively deploying AI and provides a framework for evaluating opportunities within your own organization and a roadmap for implementing AI to increase your likelihood of tangible and measurable ROI.
Foundational Elements in AI
There are a foundational set of capabilities or “skills” built into AI which are important to understand to best see how the technology can be applied to business problems. Think of them as tools in your AI toolbox that address different types of use cases. These tools can be combined and applied to create workflows where data can be enhanced in several different ways to address a particular business challenge.
- Classification and Regression - Tasks where the goal is to take input or a document and use it to predict either a specific category or a number; e.g., looking at a tweet and determining sentiment (classification) or a 1-5-star rating (regression). Classification and regression problems are algorithmically simple compared to the other categories.
- Unsupervised Discovery - In some cases, you may not know exactly what is in your enterprise data or what you want to do with it, but you’re interested in sifting through it (e.g., customer support logs) to see what you can learn. There is a lack of quantitative rigor here because the desired output is unknown, but you likely have some sense and can either confirm or deny these suspicions using this capability in AI.
- Comparison – This is for obtaining an intuitive “distance” between pieces of content as well as making a classification based on multiple documents; e.g., comparing a resume to a job description and determining how well the candidate fits the open position.
- Extraction - Extraction, as the name suggests, is useful when there is a particular sub-class of information within your content that you are trying to pull out. Typically you are extracting a class of content that can have different values that you will use in some downstream application. Sometimes it can be something large, like a whole paragraph or clause within a set of contracts. Sometimes it can be small, like a date or a product name within a form.
- Sequence Generation - In each of the preceding four tasks the output is structured, and the final goal is either a set of classes, or a concrete action. These are well-established AI problems; however, they are not all-encompassing. There is a fifth class called “sequence generation” which uses a source document to create another sequence of text. Translation is a good example. This is a fascinating field that will doubtless see massive application in the future. However, the current implementation difficulties combined with the massive size of the required datasets make sequence generation a challenging endeavor.
Data Access & Training Data
None of the capabilities above address the issue of data access. Many use cases may be based on data that you do not own or can’t get access to. AI may be the solution to your problem, but every AI algorithm requires training data to operate.
Not just any data though - you need a well-formed set of inputs and outputs of sufficient volume to make even the most basic machine learning problems tractable. And most importantly, your training data must provide quality examples of the outcome you are looking to achieve.
Fundamentally, this data is the framing of the problem and its quality and volume together will determine your chances for success in applying AI to your business. For example, let’s say that you want to develop an AI model for sentiment analysis. You’re looking to give your algorithm a piece of text – say a tweet – and have it produce a score representing whether the given tweet is positive or negative.
In order to create the model, you need a “corpus” of tweets with their sentiment labeled. How many is hard to say definitively. Simpler tasks require less data, and text tasks generally require more data than image tasks, but these are only approximate rules of thumb.
A standard dataset size for training a model would be between 10,000 and 100,000 labeled examples. Additionally, you cannot generate this data by using heuristics otherwise your machine learning model will only learn those heuristics and not the underlying behavior. It must be labeled by human hands if you want your algorithm to function. This is typically where many users “hit the wall.” Fortunately, newer approaches to AI using frameworks such as Transfer Learning, are dramatically reducing the amount of training data required - in some cases, to just a few dozen examples.
Data Science and Line of Business Collaboration
One of the key challenges in implementing any new technology is enabling business users and technical users to collaborate effectively to produce the desired outcomes. Business users need appropriate technical context to deliver the necessary inputs and technologists need appropriate business context to drive implementation decisions. This is especially true with AI where the subject matter experts (SMEs) play such a critical role in the definition of success, and the underlying technology is so highly complex.
With AI, the importance of data science expertise is typically well understood. However, the role of the business SME is often undervalued. The SME can define the task to be augmented or automated, provide valuable input in the form of training data, and help define the desired outcome and ROI. It is important to ensure that these SMEs are available for feedback during the development process.
Identifying High-Potential Use Cases
The most vital component for realizing ROI is a clear understanding and definition of a desired outcome. This enables the project team to work backwards in terms of identifying the steps that can be augmented, enhanced or automated, the data available, and a set of previously identifiable outcomes that can be used for training the models.
Part of this involves determining how accurate the process must be in order to generate a successful outcome. A common misconception is that deploying AI is a binary decision -- it will either work or won’t work. The reality is somewhere in the middle. Depending on the quality and amount of data, the homogeneity of the process, and other factors, enterprise AI solutions will deliver a range of “accuracy.”
The cost of mitigating errors in the process need to be considered when calculating ROI. A good upfront consideration is whether your goal is to fully automate a process, often referred to as “straight-through processing” or augment a process, where there will still be “humans in the loop.” These carry different considerations and ROI implications.
The use case ROI is driven by an understanding of the goals of the use case. These typically take one of the following forms:
1. Accelerating Existing Transactional Processes and Workflows
In the simplest case, the process being augmented or automated already has a transactional nature and a specific dollar amount tied to it in the form of hours of labor or process “cycle time.” These can be translated into a set of hard costs or opportunity costs. The ROI can be calculated by a goal set at the start which might take the form of “if we can reduce the hours of labor or cycle time by 80%, we will save $X of labor costs or increase revenue opportunities by $Y.” The ROI is the dollars of accrued benefit divided by the cost of building and operating an AI-based process. It is rare that any AI-based approach will be able to completely automate an existing process, but rather is likely to augment a large portion of it. The cost of the final human review of the process needs to be factored into the ROI analysis.
2. Increasing Capacity for Overburdened Processes
A good example of this type of application is customer service. There is often a significant investment in the triage of inbound customer service requests and other forms of inbound communications. AI is particularly good at classifying and “routing” this kind of content. The ROI here in addition to expanding the capacity of the current process (one agent can now handle X% more volume), is the impact on customer satisfaction from having requests handled more rapidly and more accurately.
3. Enhancing Existing Products/Creation of New Products
This is the most challenging use case to perform an ROI calculation for, because at a fundamental level this is a speculative product improvement that was previously infeasible due to its large labor cost and therefore lacks a good comparison. Another approach to justify the ROI in cases where net new functionality is being offered is to calculate what the price of such a process would be if it were implemented completely manually. While it’s disingenuous to claim any improvement here as true ROI, it can be very helpful in determining the amount of leverage your firm is deriving from AI. A product offering that would take $10m annually to replicate is likely to have more intrinsic value than one that would take only $1m annually to replicate.
If we put aside all the hype around AI, we’ve seen tremendous progress in the ability to deploy it to drive very valuable business results. If we understand the real capabilities of these tools, and work with the business to identify the right use cases to apply them against, we can generate significant ROI in a tangible and practical way.