Business processes can be incredibly hard to fathom. The more complex they are, the more difficult it is to find the magic blend of tasks, roles, flows, and other factors that distinguish a well-tuned process from a miserable flop. Even the people who’ve been part of the process for years may have little clue. It’s not just that they refuse to look beyond their job-specific perspectives, for fear of jeopardizing their careers. It’s often an issue of them being too close to the problem to see it clearly, even if they try very hard.

Process analytics is all about identifying what works and doesn’t work. It’s a key focus for us here at Forrester, and I’m collaborating with one of our leading business process management (BPM) analysts, Clay Richardson,  on research into this important topic. The first order of business for us is to identify the full range of enabling infrastructure and tools for tracking, exploring, and analyzing a wide range of workflows. It’s clear that this must include, at the very least, business activity monitoring (BAM)  tools, which roll up key process metrics into visual business intelligence (BI)-style dashboards for operational process managers. Likewise, historical process metrics should be available to the business analysts who design and optimize workflows. And each user should have access to whatever current key performance indicators are relevant to the roles they perform within one or more processes.

But if traditional BAM were sufficient for process optimization, then the core challenge would be to integrate the BAM tools native to the full range of process platforms in your organization (I’ve referred to that as the “uber-BAM” scenario, which is much easier stated than implemented in today’s hyper-fragmented enterprise process environments). However, BAM by itself isn’t enough, because it usually only aggregates historical and current process metrics, while skimping on the future-oriented scenario modeling essential to full process optimization. And, dare I say, Uber-BAM might be even worse, because it could overwhelm process designers, managers, and participants with zillions of metrics. Essentially, an overstuffed BAM dashboard is a big old haystack within which needles—the process issues—lie buried, threatening to draw blood if not detected and removed.

It seems to me that process analytics would benefit from data mining, a mature discipline that focuses on finding non-obvious patterns in deep, complex historical data sets. I’ve just completed Forrester’s first-ever Wave on predictive analytics and data mining (PA/DM), and I’ve noticed that such tools are rarely used for analyzing process issues or building predictive models of alternative workflow designs. For the most part, PA/DM remains the province of customer analytics for sales, marketing, churn analysis, and other revenue-producing activities, rather than the inward-facing, operational focus of much BPM.

Ideally, companies everywhere should be maintaining analytical data marts that store process metrics for the past several years, aggregated through their BAM tools, so that they can mine for deep-seated problems and model alternate process fixes. I call this “process mining,” and believe it should be a core feature of all BAM environments. I’m a bit disappointed that today’s BAM tools largely lack process mining capabilities. Yes, BAM tools offer varying degrees of what-if process modeling, but little of it leverages the wealth of statistical and mathematical approaches we find in today’s leading PA/DM solutions.

Process mining will definitely be among the topics that Clay and I develop in our research later this year and beyond.  We’ll also explore how such modeling capabilities could be made user-friendly for the legions of business analysts who would be their primary users. As I noted in a recent blog post, a growing range of user-friendly predictive modeling tools are coming to market in 2010, many of them integral to companies’ BI environments.

To the extent that BAM is simply an application of BI to BPM, then statistics-driven predictive modeling should be available for process analytics as well.

James Kobielus also blogs at