Building a data warehouse (DW) in today's new global economy can be very challenging. Though the conceptual DW design and architecture of a traditional brick-and-mortar company versus a dot-com may be very similar, it is the complex and dynamic nature of the dot-com organizational environment that can be the greatest challenge.

Explosive Growth

Internet-based companies, by definition, typically function in an environment of volatility, both within their operations and with the technology they use to keep up with business growth and market demand. They operate at an accelerated pace in response to the ever-changing business environment, competition and customer demands. To gain competitive advantage, dot-coms must be willing to take risks, implement bleeding-edge technology, change direction quickly and push existing technology to the limits – all important considerations and challenges for DW.

Also, executives at these organizations recognize the importance of information; they understand that in a time of fierce competition, the company that is the most effective at leveraging information will pull ahead of its competitors. As a result, these firms are beginning to plan for a DW early in the start-up and launch stages of the company when growth, complexity, change and uncertainty are most prevalent.

Unfortunately, when a dot-com gathers the DW's requirements during its start-up phase, it may not have the luxury of any source systems or data to drive those requirements. The requirements gathered for the DW may then provide the requirements for source systems, a bit of a role reversal from the traditional approach.

During the life cycle of a DW implementation, project deliverables must be flexible and extensible, changing virtually on a weekly basis as the business grows and operational systems are enhanced or upgraded with the latest technologies. This rapidly changing environment, as well as the dot-com's need to respond quickly to new business opportunities, requires that the DW implementation team be extremely resourceful and adaptable. The explosive organizational growth within a short period of time coupled with a rapidly changing operational environment makes DW planning and sizing complex.

Another important challenge that we are seeing associated with Internet start-ups is the "need for speed". The demand for rapid implementation and quick results contributes to extremely aggressive schedules and short implementation windows that do not allow for mistakes or miscalculations. The DW has become an important enabler for dot-coms. As such, the DW must be in place and must be flexible and scalable enough to keep up with the pace of the business. Because of this, Internet companies are very interested in prebuilt analytical applications supported by a data warehousing model and in technologies that allow them to get their DW up and running in an accelerated time frame.

At the same time, these firms understand the importance and logic of supporting an iterative approach to data warehousing. This is the best way to balance and achieve their requirements relative to timing, flexibility, extensibility, scalability, new technologies, etc. As such, dot-coms typically prefer multiple product releases in short intervals versus the single release (big-bang) approach, even if all requirements are known up front which is rarely the case. Although the implementation window is extremely small, the speed with which the business changes, as well as the dot- com's willingness to use new technology, forces change in all phases of the DW implementation project. A sound change control process and a highly integrated team with the ability to provide high-quality work and deliverables on accelerated time-tables are, therefore, necessary for a successful implementation.

The Dot-Com Organization

Dot- coms typically contain a group of IT zealots who are technically advanced, extremely demanding and in short supply. Working at a brisk pace to develop tools, warehouses, systems and processes that support their organization's business and competitive needs, these professionals must be both capable and resourceful. It is not uncommon for professionals in this type of position to work extremely long hours – sometimes through the night – to enter new markets and expand the business "footprint".

This type of working environment places pressure on the DW implementation team to meet the ever-changing business and user requirements, and to remain informed about the strategic and tactical business implications to the DW design.

On a positive note, many executives are technically savvy individuals. They are becoming increasingly aware of recent technology advancements; the value of information; and how, when integrated through a DW, the flow of useful analytics related to customers, products, competitors, vendors and markets for competitive advantage can be increased. These executives are willing to assume risks to implement a new technology solution; and it is not unusual for them to expect immediate results – another challenge for the DW implementation team.

Technical Architecture and Implementation

In the high- growth and dynamic dot-com environment, the architecture of the DW must be both highly scalable and flexible to accommodate new subject areas and operational systems with minimal impact to the existing data warehouse environment. It must also be highly sophisticated in order to address significantly higher data volume associated with clickstream data, which is different from the traditional data transformation associated with legacy systems. The ETL (extract/transform/load) component of a dot-com requires building a DW architecture that addresses the operational problems of many diverse source systems, including new sources of information such as the Web, syndicated data and wireless technologies.

What Does This Mean to the DW Professional?

The next generation of data warehousing will be driven by a few important themes: time to market; the tremendous increase in data volume brought about by the need to store granular customer information including clickstream data and e-analytics; and the need to make this information available to all customer touchpoints. Therefore, DW practitioners and product vendors must leverage their experience, methods, industry expertise and technology to develop prebuilt, integrated knowledge-based solutions and analytical applications to be successful within the "virtual walls" of today's dot-coms. Furthermore, to create a DW that acts as the enabling foundation and leverages information effectively throughout the organization in order to create sustained competitive advantage, the implementation team must combine both business and technical skills. At the same time, the team must be creative, flexible and – most of all – adaptable to working in relatively unstructured environments where change is the norm.

While new-economy companies certainly present some unique challenges for these individuals, the satisfaction that can be derived from implementing data warehouses that deliver real value to start-up enterprises is unparalleled.

Gayle Greenlee and Sunil Bagai, principal consultants in PricewaterhouseCoopers' Global Data Warehousing Practice, contributed to this month's column.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access