In August 2002, Aberdeen Group did an informal survey of Web analytics users, Guy Creese wrote in a recent Aberdeen Insight. The companies surveyed included enterprises in a variety of businesses: food, retail, consulting, software and entertainment were among those represented. This small sample uses software or services to understand Web site visitor behavior. The companies are relatively happy with their vendors – 10 out of 12 felt that their current vendor met, exceeded, or greatly exceeded their overall expectations. However, the satisfaction did not come easily. A number of them are on their second or third solution, having de-installed well-known names such as Accrue, Keylime and WebTrends.

One user summarized his experience of Web analytics: “[A] complete disconnect between promised results and delivered results from some vendors. Web analytics is part science and part art – this opens the door to some extreme exaggerations on the part of some vendors struggling to stay alive.”

This Aberdeen Impact draws on this survey of Web analytics users, as well as numerous other interviews, to describe, at a high level, what they are looking for, as well as their frustrations with the discipline.

When the Aberdeen survey asked these companies what were their top five buying criteria, they replied:

  1. Accuracy of results
  2. Timeliness of results
  3. Scalability
  4. Total cost of deployment
  5. Custom reporting/manual segmentation

Enterprise users gave vendors high marks for delivering four out of their desired top five buying criteria:

  1. Custom reporting/manual segmentation
  2. Accuracy of results
  3. Scalability
  4. Total cost of deployment

However, timeliness of reports was missing from the top five. That capability hovered around the middle in terms of vendor ability. This suggests that browser-based tracking will continue to be popular with enterprises because the technique enables quick report turnaround by avoiding time-consuming Web log transformation.
A result of being superb at custom reporting is that vendors are overwhelming users with extraneous information. More is actually less. As one user noted, “Really understand how you will use this data. Many reports look and sound nice but have no usefulness in analyzing or marketing the site.”

This sense of irrelevance was a common complaint. A different user noted, “Identify a core set of business metrics and focus your efforts on running regular analysis based on those objectives. At some point, you hit a threshold of potentially ‘overanalyzing,’ which doesn't necessarily give you…[a] clearer picture.”

Said another user: “Work with a consultant or experienced implementer. Plan it out. Work through in advance what you want to do with the results. It’s easy to just want everything, and ‘every-thing’ leads to paralysis.”

Given the difficulty of Web analytics, it is not surprising that practioners recommend kicking the tires. One user noted, “Cross-compare at least three suppliers in detail and ensure you have a demo using your own live data – especially if going for a custom build.” Another said, “Focus upon the vendor’s ability to merge disparate data sources into one report (Web logs, personal profiles, e-commerce, etc.). Not many vendors can do it and within the right price range.”

Aberdeen conclusions: Although not yet a crisis, all is not well in the land of Web analytics. Enterprises have had to make several tries to find a package they like, and they still feel that the discipline is more difficult than it needs to be.

These findings of disquiet are provocative and elicit more questions than they answer. Consequently, Aberdeen, with the help of Experient and in concert with EContent Magazine, is embarking on a much larger survey. This survey on the current state of Web analytics will also access how quickly enterprises plan to move toward an e-channel awareness suite, a phrase that Aberdeen has coined for the consolidation of Web analytics with content management, search/categorization and personalization into a business process.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access