One of the most important developments in data warehousing in the new millennium is the advent of the confluence of data warehousing and ERP. These two gigantic movements have been developing independently for years. Both have great momentum, and the only surprising thing is that it has taken so long for them to converge.
There have been some good reasons for the length of the convergence period. The origins of ERP have been decidedly operational transaction processing. The origins of data warehousing have been decision support. It is as if one movement has been country and the other has been city, or one movement has been rock-and-roll and the other has been classical. Both movements have existed in the world in parallel tracks for a long time with little or no crossover until now.
As ERP systems have reached the implementation state (as a result of Y2K), the discovery is being made that implementing ERP only gets the data into the system. In no way does the implementation of ERP prepare data or information for use and analysis. The very architecture of ERP is contrary to the needs of access and analysis. This is the point at which the ERP vendors have discovered (or are currently discovering) the need for data warehousing.
Data warehousing usually comes as a complete culture shock to the ERP vendors. The rules of the road that have been carefully learned in doing transaction processing are all destroyed and turned upside down by the notion of data warehousing. Whole careers spent understanding application development are negated. As anyone who has done data warehousing could have told the ERP vendors, the last person you want building the data warehouse is the successful developer of your applications, especially your transaction-processing applications.
All of this comes as a dip into ice cold water for the ERP vendors. But in the long run, the cold dip is something that is both cleansing and invigorating while initially very shocking and painful.
From the larger perspective of architecture and the industry, where is everyone heading? Figure 1 illustrates three major possibilities for the industry's direction.
Figure 1: Possible ERP/Data Warehouse Architectures
The first possibility, Scenario A, is one where applications outside of the ERP environment will place their data inside the data warehousing facilities of the ERP vendor. Of course, this scenario is the one ERP vendors are hoping for.
Scenario B is one where independent data warehouses will be built. The non-ERP applications will build their data warehouses in standard software such as Oracle, DB2, Teradata, Informix or others. The ERP data warehouse will be built in the facilities provided by the ERP vendor. This scenario is very likely to be seen later.
The third scenario, Scenario C, is one in which data is pulled out of the ERP environment into a standard data warehouse technology. There is a movement toward this direction today with the advent of technology that can read and access ERP data and pull the data out of the ERP environment. Companies such as Acta, SAS and Informatica are finding that this technology is proving to be very popular among shops that are tired of listening to promises from an ERP vendor that has never understood a data warehouse.
These three scenarios will all happen. It is only a question of degree. Will the future hold the most promise for Scenario A, B or C? Will more companies choose Scenario A over B and C? In order to predict the future, it is best to analyze the perceived advantages and disadvantages of each scenario.
In Scenario A, all data will flow into the data warehouse facilities of the ERP vendor. There are ERP vendors that do have a very appealing and enlightened approach to data warehousing. For those ERP vendors, Scenario A is a real possibility. Also, when it comes to understanding the ERP vendors' data, who would know it better than the ERP vendors themselves?
The disadvantages of having an all-ERP world for data warehousing are many:
- Some ERP vendors have never shown they have data warehouse expertise. Unfortunately, ERP vendors are experts in transaction processing, not DSS processing.
- Some ERP vendors require you to convert non-ERP data into ERP. This path is very fraught with problems. The past track record of ERP vendors shows bringing anything into ERP has been expensive. Bringing legacy application data out of the operational environment is so difficult that an entire industry has sprung up (the ETL industry), and it is very difficult to bring data out of the application environment whether you are bringing it into ERP or not.
- While ERP has been implemented in one part of the organization, the data warehouse effort may already be quite mature in other parts of the organization. Organizations that already have a mature and well-populated warehouse are not about to allow the warehouse to fall into the hands of the ERP vendor.
- The ERP vendor has a fascination with inventing everything themselves. The ERP vendor will not allow access and analysis vendors to directly and freely access ERP data. Instead, ERP vendors insist on editing and controlling the access to their data. No person who has had free access to data trusts the ERP vendors based on their past track record of lack of freedom to access of data.
Scenario B is one in which there is a data warehouse for non-ERP data and a separate data warehouse for ERP data. This scenario is likely for many environments.
The advantages are that this scenario is a "natural" one. There is already evidence that many organizations are starting down this path. Data warehousing and ERP have been evolving for a long time now, and Scenario B is a natural path to take. The scenario fits organizationally and politically. The scenario also is a natural technological fit. For all of these reasons, there is a strong impetus toward Scenario B.
The disadvantage is that there is no true corporate perspective that can be achieved. Data resides in one place or the other. The limit of the perspective is that contained in any warehouse. Of course, the argument can be made that a consolidated view of the data can be created by combining data from the two warehouses. But, in truth, such a sharing of data is never a practical or efficient event. There will be two environments, each of which have their boundaries. The fulfillment of an enterprise perspective of data cannot be achieved in this scenario, but much free and unfettered DSS analysis can occur.
Scenario C occurs when an organization grows tired of the promises made by the ERP vendors and takes matters in its own hands. Scenario C is a very valid scenario for the adventuresome and independent-minded organization. In Scenario C, there is true freedom from the ERP vendor, true enterprise integration and the opportunity to use and exploit the rich world of data warehouse tools and software.
The advantages of this scenario are obvious. The disadvantages of this scenario are that data must be weaned from the ERP environment. Fortunately, there is technology (Acta, SAS and Informix) for this task. It becomes the task of the third-party vendor to keep up with changes in ERP. But that's okay because these vendors have a long track record in doing just that.
Every corporation will have a certain propensity toward one of these scenarios based on their current structure and past history. The key is selecting one of these scenarios so that you can reap the benefits of data warehousing.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access