Radha R would like to thank Krishnan Raman, a general manager with the business intelligence practice at MindTree Consulting, for contributing this month’s column.


In the current competitive business environment, a successful data warehousing organization may well be the successful business enterprises of the future. A reflection on the millions of dollars spent on business intelligence/data warehousing (BI/DW) solutions by individual organizations, that has yielded less than expected ROI and the burgeoning TCO for these organizations to maintain those solutions is enough to make us focus on what needs to be done better. Even in a traditional factory setup, there is a practice of reviewing the health of the equipment by monitoring critical parameters like vibration, noise-level, wear and tear limits etc., as well as a rigorously adhered to preventive maintenance schedule. However, I find that a DW solution that has multiple choreographed moving parts, with an immense potential to deteriorate is seldom considered a candidate for a routine health-check.


When implementing DW solutions, you need to keep revalidating the strategy as you progress, depending on the environmental factors. Big bang implementations have been massive failures. You typically adopt a phased roadmap BI/DW solution, which can be even tougher as there is no static goal to be achieved. The goal for a BI/DW solution is continuously shifting:

  • Business needs to be served by the DW can change with changes in the business processes.
  • Changes to an environment like a merger, acquisition or a restructuring can introduce radically different requirements.
  • Emerging technologies and growing maturity require recalibration of strategy. An organization that was comfortable with basic reporting and analytics, would want to mature over a period of time and move up the continuum towards dashboards/scorecards, data mining, predictive analytics and operational BI.

With DW being one of most crucial and a “financial big spend” implementation for every organization, it is critical to learn from the past and leverage best practice approaches and avoid pitfalls.


What is BI/DW Assessment?


A DW initiative is a journey with a clear goal, but its undertaken in phases with appropriate checks and balances.. Your enterprise must understand and commit the people, processes and technologies necessary to achieve an enterprise DW solution that manages data as your corporate asset. The essence of DW assessment – the why, what and how of assessment – is directed at defining and refining the warehousing process to place the DW initiative in the right trajectory to successfully achieve the overall business goals.


Why Assess the BI/DW Solution?


The reason for assessing a DW depends on the type of assessment. Based on my experience, I would classify assessments into two broad types:

  • Readiness assessments, and
  • Post implementation assessments.

Any new DW initiative will benefit from a rigorous and comprehensive assessment. This will help understand the current landscape, set the right goal and position the initiative for success.


A more prevalent assessment is in an organization with an existing DW environment. A DW solution is a long journey and you tend to loose steam and waiver in your direction over a period in time. There are multiple causes that warrant an analysis and a renewed life cycle for the implementation:

  • The existing solution is not providing value due to fundamental flaws in conceptualization.
  • Program needs re-alignment to environmental changes.

The Figure 1 captures some of the critical reasons for performing an assessment of your BI/DW solution.



What to Assess in Your BI/DW Solution


There are four critical areas that need to be reviewed as a part of assessments.


  • Business need coverage. Business need assessment includes an analysis of the underlying business drivers, objectives and overall context of the business need that has been established for analytical purposes. This helps identify current business issues and gaps in the existing solution leading to a quantitative and qualitative feel on the capability of the solution and the value perception on the benefits and ROI accrued from the solution. Maturity of the solution implemented is important to be understood (on the maturity continuum of basic analytics, dashboard/scorecard implementations, data mining, predictive analytics) to craft a solution that will push the organization up the maturity curve.   
  • Data Model Flexibility. This includes an analysis of logical data structures, their feasibility, completeness, documentation and fit to business requirements. A detailed analysis of how well the metadata (conformance and documentation of various measure, dimension and hierarchy definitions) is maintained in the organization is important to be achieved here. An assessment is also done through a structured scenario analysis approach on the nimbleness of the data model to adapt to changing business requirements (say a merger, an acquisition or a restructure) and the scalability/flexibility of the data model to emerging requirements (e.g., need to move to a data mining solution from base analytics).
  • Technical scalability. Technical evaluation is focused on assessing the current underlying hardware, software and network infrastructure. Also examines the physical database designs. Performance scaling issues due to size of data bloating and proliferation of independent tools demanding more infrastructure are common concerns seen. In one of my engagements, I found that there were fourteen different tools just on the reporting side and it had driven the TCO through the roof by complicating the maintenance. DW/BI is an area that is still continuously evolving from a tool maturity perspective and you can have a situation where the tool vendor’s consistent version changes introduce risk to the implementation.
  • Process and organization effectiveness. Process evaluation is focused on the effectiveness and completeness of the processes being adopted as a part of development and ongoing support of the solution. One area that is often compromised and shackles scalability of a DW solution and needs critical evaluation is the level and sanity of documentation on standards and the solution. Other factors include project communication, decision making/responsibility structures, change management and issue resolution processes, data stewardship processes, configuration management processes, etc.

How to Assess a BI/DW Solution?


Enterprise data warehouses (EDW) are typically large and complex solutions, and so is the process of assessing them. These solutions typically cut across multiple functional areas within the organizations attempting to bring in certain amounts of standardization and transparency across the organization. It attempts to tear down the protection walls that people have created over a long period of time among functional domains. It also surfaces the inherent divide that we find between the functional and the IT sides of the organizations. Thus, there are a few critical considerations to address prior to embarking on a BI/DW assessment.

  • Managing politics. DW solutions as mentioned above tend to be operating in sensitive situations. Therefore, assessment teams need to be capable in terms of managing the politics between the various stakeholders. There is a two-fold approach to address this:
    1. Having a strong team capable of managing such sensitive environments, and
    2. Being absolutely rigorous in your data collection and logical synthesis.

    It also dictates the need to maintain neutrality and objectivity in the assessments. Regardless of whether the business or IT function initiated the assessment, it must be performed from the perspective of information as an asset to the enterprise.

  • Managing complexity.Many times, I come across “A Teams” that get lost in the complexity by delving themselves into too much of the details. In these situations, it is important to employ a sufficient amount of synthesis to arrive at a solution, but mathematical or quantitative perfection is not the objective.
  • Managing expectations. While organizations appreciate the logical value assessments, the maturity to sign-off a budget for an assessment is low. Hence, they start looking at expanding the scope and expect the assessment to produce exhaustive business requirements. While it is possible, considering the tight timelines that assessments are typically expected to adhere, it is best avoided.
  • Considering best practices. A good approach to manage internal differences and diffusing cross-functional positions, is to leverage industry best practices while recommending solutions. It tends to be less controversial and if an external party is present, it makes the decision-making process smoother.

Building on these core considerations, and the environmental situation prevalent in the organization, we need to adopt a fine-tuned assessment approach for the organization.What usually works is a combination of structured and somewhat unstructured approaches. On the structured side, we need to have facilitated group sessions, questionnaires and interviews. In any of these sessions, the script should not be too restraining and should provide flexibility to explore based on the responses to the unstructured aspects. Asking the right questions at the right time is the basis for successful assessments and leading questions provide a basic check list to advance. You need to be prepared to drill down as far as necessary to obtain the needed information. Leading questions are often the start and the answers point the way to identifying weaknesses and proposing improvements.


DW implementations require speed, agility and extensive dynamic planning. Assessments in DW implementations are more like a pit stop, where you recoup and do dynamic planning to ensure success in the future. Ignoring this pit stop puts your DW implementation at peril!

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access