As the mobile Web grows in popularity, so does the need for mobile measurement. When put to the test, with a basic mobile campaign, we find that mobile measurement has not yet reached the level of maturity that we have come to expect from traditional Web analytics.

It should be noted that the observations presented here are based on recent real-world experience doing mobile analytics. The site we analyzed – although it must remain anonymous – is owned by a major corporation and exists to promote the download and purchase of mobile games. The problems we encountered as we developed a measurement framework will likely cause similar trouble for anyone who follows in our footsteps. Treat this as a cautionary tale, and also as an exposition of the unavoidable shortcomings of mobile analytics as it currently exists in practice.

Limitations of Mobile Data Collection

If we treat mobile measurement like traditional Web measurement, we face challenges right away, starting with data collection.

Many popular Web analytics tools available today – like Omniture and Google Analytics – rely solely on JavaScript tags for data collection. Using this method, site owners place a tag on every page they wish to track; when a visitor accesses a tagged page, event-level information is sent back to the Web analytics tool. That, in a nutshell, is page tagging.

Tracking mobile Web in this manner is problematic, though, because mobile browsers do not reliably support JavaScript. Some tag-based Web analytics tools provide a fallback tracking method when JavaScript is not present – namely, a hard-coded 1x1 pixel image request that collects a subset of the standard event-level data.

An increasing number of mobile-specific tracking solutions are surfacing on the market, and they offer arguably more robust data collection methods than traditional Web analytics tools. However, despite known deficiencies, site owners may still opt to use a traditional Web analytics tool for mobile measurement simply because it’s already an accepted business standard for tracking existing Web content.

In this article, I will outline some of the challenges faced when using a tag-based Web analytics tool adapted by way of an image request to measure mobile.

An Example

Consider two very similar online campaigns aimed at promoting the download of an application. The first campaign goes out as an email blast on traditional Web, and the second campaign goes out as an short message server blast on mobile. Beyond the difference in screen size, these two campaigns look a lot alike: a message, a click, a download. But, when it comes to measuring the effectiveness of each campaign, mobile brings a whole new set of challenges.

Using the pair of simple campaigns as an example, I will illustrate common mobile measurement issues concerning campaign clickthroughs and downloadable files.

How do we Track Campaign Clickthroughs?

In our campaigns, site-side tracking begins the moment someone clicks through from the message he or she receives to the landing page on the site. This tracking process is well-defined on traditional Web but less so on mobile.

Campaign clickthroughs are typically tracked using a code appended to the query string of each URL. When a visitor clicks through to the site, the campaign code gets collected by the JavaScript tag on the landing page and is then passed on to the Web analytics tool. Without the use of JavaScript, a bit more effort is required.

On mobile, and with image requests for data collection, we cannot easily intercept campaign code query parameters on URLs. As a workaround, unique landing pages can be set up to track traffic from each campaign. This approach, although it works, adds a layer of complexity to the campaign creation process beyond what we would normally do to track campaigns on traditional Web sites.

How do we Track Downloads?

The measure of success in our campaigns is the download of a file. Once again, established tracking techniques on the traditional Web must be re-evaluated on mobile.

Traditional tag-based Web analytics tools track downloads at the moment the visitor clicks on a link to get a file. This counts downloads initiated, not downloads completed, but it is a concession we’re willing to make in return for the convenience of tag-based tracking. However, because link click tracking requires the use of JavaScript, even this limited functionality is lacking on mobile.

On mobile, and with image requests for data collection, we must make do by treating the page just prior to download as a signifier of the download itself. This method will necessarily inflate the number of downloads, though, because it’s entirely possible to back out of the process before actually requesting the file. In this case, clear communication to data consumers is recommended in order to set expectations about data quality.

After All That, Just the Basics

The tracking techniques described above provide the absolute bare minimum we need in order to measure mobile campaign effectiveness, namely, page views of the campaign landing page and downloads of the file. Tracking becomes more problematic if we hope to get accurate visit and unique visitor counts; mobile browsers do not reliably set cookies, and we are left with very weak rules for visitor identification.

In comparing very similar activities across traditional Web and mobile, the lesson learned is that mobile requires somewhat more effort just to get the most basic tracking in place. While we need not reinvent the wheel as we move into this new medium, mobile forces us to reconsider some of our fundamental assumptions about Web measurement.

What Does the Future Hold?

Although the current state of mobile analytics leaves much to be desired, there is certainly hope for the future. Just as traditional Web analytics tools and techniques have become more robust with maturity, the same will inevitably hold true for mobile analytics.

The evolution of mobile analytics will likely be driven by two separate but related factors: mobile device technology and mobile analytics technology. As more phones on the market run JavaScript and accept cookies, our ability to collect data will improve; as more businesses strive to build mobile sites and track mobile behavior, the tools and techniques we employ for mobile analytics will improve.

Web analysts view the maturation of mobile devices with anticipation in part because it requires no action on behalf of the analysts (other than waiting a couple of years). When – eventually – the majority of mobile devices run JavaScript and accept cookies, we will be able to use page tags to get the same level of tracking on mobile Web sites that we do on traditional Web sites, and with the same level of effort.

In tandem with this development we can expect traditional Web analytics vendors to expand their support for mobile Web analytics. Within traditional Web analytics tools we increasingly see new reports for mobile-specific metadata such as mobile device types and manufacturers, a well as improved control over mobile visitor identification methods.

Additionally, as more Web analytics practitioners implement mobile tracking, the collective body of knowledge about mobile analytics best practices will continue to grow. As with traditional Web analytics, this increased adoption will subsequently drive further improvements in the tools that we use.

As a result of the improvements mentioned above, we should eventually be able to use a unified Web analytics solution to adequately track both mobile and traditional Web sites. In the meantime, we either make do with our existing analytics tool's limited functionality, we opt for a separate mobile-specific tool or (obviously the least desirable choice) we don't measure at all.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access