Petabytes (1,000 terabytes) of data are clogging data marts and warehouses around the world. The amount of data doubles every year. IBM predicts that within a few years, data will double every 11 hours. Wal-Mart purportedly accumulates one terabyte of transactional data every day. Soon we'll be talking about exabytes, zettabytes and yottabytes, each a thousand times larger than the previous.

Companies face the challenge of how to store their oceans of data. For certain companies, the best option is to aggregate days' data into weeks, weeks' data into months and months' data into quarters. Others prefer to delete. In fact, certain executives recommend that only two years of data is viable; beyond that time, they say, data has no value.

Yet legions of corporations are using weekly data that is years old to compete and beat competitors at winning customers profitably. The age-old problem of what's working and what's not among the many marketing activities that a company sponsors in a given week is no longer a mystery when past data is analyzed.

The most critical use of data in the marketing arena is econometrics, which is the statistical analysis used to determine the levels of factors that best explains a result. For example, consider the question, "Do TV ads generate revenue?" Econometric studies work best with three years of weekly marketing data. Without three years of data, results are less detailed. In addition to a sufficient time span, the data being measured needs to vary over time. Variation is a requirement for comparing the inputs to the output side of the equation, so companies preparing to use econometrics to determine the optimal marketing mix should ensure that data is weekly and varies over the study period.

Increasing sales is not the first goal of every company. Some businesses are more focused on increasing profits, customer lifetime value or customer equity - all bottom-line goals. Other companies are interested in unit sales, market share or customer satisfaction, which are not as valuable when optimizing a budget, since top-line goals don't necessarily have a monetary value.

Figuring out what's working and what's not can be a major headache. Add to this problem the economy and outside factors, which further complicate the determination of what's driving sales.Econometrics calculates an equation built from historical data. On one side of the equation is the goal or the dependent variable (see Figure 1). On the other side of the equation, independent variables include the marketing spend for traditional and new media, sponsorship, direct marketing, etc. Independent variables also include economic factors, competitor activities and some variables with no monetary value, such as social media and public relations.

As an art and a science, econometrics examines the inputs (independent variables) each week and the outputs (dependent variables) during the same week and following weeks for longer-term effects. By analyzing input and outputs every week over a three-year period, the numerous marketing levels occurring over time provide a sufficient variance for a comprehensive analysis. The resulting equation of calculated results mimics actual results and predicts future market response of activities.

While econometrics analyzes the data to determine which activities lift or impact the goal, a financial model incorporates the econometric effects (see Figure 2). The results of the analysis are then used to establish a marketing budget and optimally allocate that budget across marketing activities, products, customer segments and regions.

Industry consolidation has spawned many legacy systems, making it harder to gather the data into a database. However, data indexing providers can assist the business by identifying specific rows and columns to include from the data mart without actually building a new database. A data indexing service strategy facilitates an easy course for grabbing data across disparate systems.

Consider how your business fits this mix of requirements. While certain companies are aggregating or even deleting data, others are keeping three years of historical weekly data to use econometrics and optimization that will finally tell companies their ideal marketing budget as well as optimal allocation in the marketing mix.