New Jersey's data warehouse initiative was launched in early 2000 with the goal of integrating and delivering accurate data from the many disparate state information systems to support decisions by staff and management across all state agencies. With hundreds of different legacy platforms and applications in use across departments, New Jersey's state agencies had been handicapped by an inability to easily or cost-effectively exchange or coordinate information across systems. This lack of coordination across state agencies translated into tangible and costly problems in terms of inefficiency, information quality and service.

Compounding these problems was New Jersey's growing use of e- business applications which provide a one-stop Internet resource for businesses. These online systems dramatically increased the volume of information filtering into the State's IT systems and added yet another layer of applications and databases onto the State's already complex environment.

Business users in each agency repeatedly asked for integrated information, but in the past there were simply too many different systems and data sources to integrate them manually in a cost-effective and manageable way.

To solve these problems, New Jersey's office of information technology created an office of data management services. This unit launched a major initiative in early 2001 to create a statewide data integration architecture that would bring together information from the State's various information sources and deliver that information to a variety of planning and analytical applications. After a survey of available solutions, Ascential Software's DataStage XE enterprise data integration platform was selected as the extract, transform and load (ETL) solution for the project based on its broad support for mainframe, UNIX and NT data sources, and its highly scalable graphical interface and processing engine. Additionally, the decision was made that the first deliverable for the project would be the creation of a data mart for the office of management and budget (OMB). OMB officials immediately recognized the value of developing such a system with an integrated architecture that would benefit other agencies. Early results from this warehouse would deliver rapid ROI on the State's overall data integration efforts and invigorate development of a second round of marts for other state agencies.

Deployed in November 2001, New Jersey's first data mart for the OMB is today used by dozens of analysts to prepare budget forecasts, identify financial trends and provide better decision support to the administration. Five nights a week, more than 300 megabytes of financial and budget data is extracted from mainframe systems and loaded into an Oracle database on an IBM AIX server. Every two weeks, an additional 500 megabytes of payroll and personnel data is extracted from the mainframe systems and integrated into the Oracle database. Analysts in the OMB use the resulting data mart to compare actual and forecasted costs with budgets and other projections. Development on additional solutions for the department of environmental protection, the department of transportation and juvenile justice commission is already underway. Six additional data marts are in the planning stages.

In addition to providing staff with better information access and improved data quality and consistency, the most important benefit of New Jersey's new statewide data integration solution will be the elimination of hundreds of redundant data extraction programs. In the past, individual agencies retrieved data for specific projects by manually writing COBOL or Focus extract programs, resulting in hundreds of redundant programs which might download the same information each week for different projects.

By replacing these manually written extract programs for the OMB project with automated reusable data integration routines and a standard statewide data model, the state of New Jersey merged more than 100 redundant extract programs and converted a three-hour weekend batch process into a 30-minute daily task. Using this integrated approach, new agencies can access the same analytical capabilities at an average savings of $500,000 per deployment over traditional methods.

We plan to repurpose freed resources toward new projects that improve service to our agencies and their constituencies. These new programs, as well as the State's actual and anticipated cost savings, are a direct result of our use of Ascential's DataStage XE data integration solution and our decision to build a statewide data integration solution based on a common data model and data architecture.

Ascential Software Corporation's DataStage family is the industry's most comprehensive enterprise data integration platform, enabling companies worldwide to create and manage infinitely scalable, complex data integration infrastructures. The DataStage platform enables successful implementation of data-intensive applications such as customer relationship management, supply chain management, enterprise resource planning, decision support and e-business.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access