Related Links

Web Seminars

In good times or bad, businesses must transform data into useful information to not only run the business, but to grow it and increase profits. In addition to sales and profitability, businesses need data to provide transparency, privacy and security for government and industry-specific regulations or initiatives. 

Why We Must Change

Although demands for information expand exponentially, businesses keep falling behind because they continue to look at data integration as an IT task. This narrow focus inhibits businesses from transforming data into the comprehensive, clean, consistent and current information they and their customers, partners, suppliers and stakeholders crave. They are losing competitive opportunities and are unable to provide transparency, leaving them perpetually behind the curve of information demands.

Unable to wait for late, over-budget IT projects, businesses rely too much on data shadow systems (or spreadmarts). They lose productive time and resources hunting, gathering and merging data when they should be focusing on running the business and increasing revenue and profit.

The Old Ways

Businesses need to take a new approach to data integration if they want cost- and resource-effective access to information. They can start by recognizing the following misconceptions: 

• Data integration is IT’s responsibility. (Reality check: data integration is only as good as the quality of the definitions of data, transformations and metrics that only the businesspeople can provide.)
• Data integration is just a coding task. (Reality check: it’s less expensive to buy a tool.)
• Data integration is only ETL. (Reality check: there are many other aspects of data integration, such as data profiling, data quality and consistency, and operational processing, that go beyond basic ETL capabilities.)
• Data integration is dealt with in a project-by-project, tactical approach. (Reality check: it should be enterprise wide, like phones and email systems.)

The Leopard Needs to Change its Spots

When things are not working, it is time to change your approach. Learn from the many companies of all sizes across various industries that have been successful in implementing data integration. These companies have broken through the misconceptions and reoriented their data integration efforts. 

It’s the Business

Restricting data integration to IT is a recipe for disappointment. Information won’t represent what is really happening in the business, forcing businesspeople to fill the information gap by building data shadow systems that sap productivity and result in poor data.

The business should be involved in three ways: sponsorship, participation and governance (see Figure 1).

First, although the business needs to sponsor or fund a data integration project, sponsorship is more than writing a check. Sponsorship means commitment of resources and time, not just for the current critical priority but on an ongoing basis. It does little good to sponsor data integration activities if your businesspeople don’t have the time or inclination to get involved in governance and in the project. If the business sponsor isn’t willing to change business processes and priorities, then his/her subordinates will get the message that it’s a waste of their time and the investment will be lost.

Second, business participation in data integration projects from beginning to end is necessary to get relevant data. This starts with gathering the business and data requirements for the project but goes much further. The most often overlooked aspect of these projects is data quality. Many people are surprised at the end of the project that data quality hasn’t improved and, in fact, more data quality problems are uncovered. IT needs to work with the business to know the current state of data quality within the source systems. More importantly, it is necessary to understand the state of data quality across business systems, especially as it applies to inconsistent product, customer and supplier data. The business must be involved in uncovering data quality, determine what to do about it and establish the priorities about what should be “fixed.”

Finally, data governance is essential both on the current project and as an ongoing activity. Only businesspeople can provide the business definition of the data, how that data needs to be transformed into useful business information and the metrics that measure business performance. Just as the business evolves, so does the accompanying business data. Unlike a fine wine, data does not age well. It needs constant attention from the business to update changes to the definitions, transformation and metrics.

It’s Not Just the Tool (but You Need One)

Hand coding remains pervasive in data integration projects because ETL tools once were expensive and hard to use. Although times have changed, it’s still a common practice for IT to crank out the SQL code for ETL. Because the tasks are complex, the SQL code stretches on for many pages for each data source, and there are numerous SQL scripts scattered across the enterprise to gather data from all the necessary data sources. Before taking this cumbersome approach, ask: