New Trends in BI Requirements-Gathering
Traditionally, business analysts on business intelligence projects have spent their days interviewing, analyzing and organizing their work in what I call a document-centric manner. The documents in question are maintained with the usual office automation software, such as word processors and spreadsheets. Business requirements are listed in Microsoft Word, and data requirements are depicted with data models that usually started as static diagrams. And for basic content management, tools like Microsoft SharePoint are often used as repositories to provide access to the different artifacts stored as files.
These traditional approaches have worked for smaller projects. However, when multiple contributors maintain a formal business requirements document (BRD), take care to prevent people from overwriting the same sections of the document. Efforts by different contributors, including business analysts, subject matter experts, data architects and metadata experts need to be tracked and merged into the final document.
Moreover, if a single business requirement appears across documents, it needs to be cross-referenced to maintain consistency. This cross-referencing is often done manually, mandating administrative vigilance when documents move or change names. In the end, the document-centric approach to requirements gathering becomes awkward, labor-intensive and costly.
New Approaches, New Tools
Emerging requirements automation software tools include mechanisms that interrelate different requirements more granularly so they can be linked across documents and projects. BRDs can appear holistically with cross-referenced objects using hyperlinks embedded in the document. These objects may be text, diagrams, screenshots or spreadsheets and may appear as if they all existed in line in the document. This capability renders a BRD richer and more illustrative than text-only requirements descriptions.
As individual BI projects become more complex and services that have shared business processes and common needs emerge, requirements automation tools can drive efficiencies. This is especially true in cases when:
- Requirements reviews need to be done with geographically dispersed business users,
- The same business requirement is common across BI projects,
- There are several business analysts and small to midsized enterprises collaborating on a BI application,
- Diverse lines of business share the same requirements and
- A centralized team of business analysts or data stewards are responsible for gathering requirements at the enterprise level.
This concurrency and complexity of projects drives the need for a single requirements repository that allows access to common artifacts. As requirements are collected over longer time periods, they can be reused and applied to future projects. And as they span enterprise-level needs, they will be not be scattered in project-specific documents but shared across business initiatives. A new crop of software tools seeks to help in integrating these requirements across projects on an ongoing basis.
A Standard Methodology
These tools establish a common methodology that standardizes an analysis process using defined templates at every step. Some methodology steps and flows are specific to each toolset; others try to fit within common industry approaches. Standardized rules to enforce mandatory fields, naming, data types and relationships can be entered by business analysts or data stewards in a sustainable way. Some tools use templates to generate dimensional data models along with metadata for business rules and calculations. This reduces development time and ensures that requirements and data structures originate from the same place.
Business requirements toolsets produce artifacts that can drive business analysis workflow, driving subsequent design and testing rigor. It's a cumulative process that ensures that each artifact leads to the next artifact, thereby enforcing the implementation standards for the project. The benefit is the adoption of a common process, a set of rules and consistency of enterprise data. The fundamental requirements goals of clarity, reusability and consistency are thus more easily achieved.
These requirements toolsets also provide a cross-functional view for business analysts in different departments. Consider the following business rule: "When an existing customer opens a new account, the customer profile is updated." This rule should apply across the customer lifecycle, touching and affecting multiple lines of business. The tool identifies the primary "owner" application and all its related usages. If a requirement changes due to a regulatory reason, the tool can make the change in its repository and cascade the relevant change to related applications.
The following is an example of how one software tool provided a systematic progressive linkage of documents on one of our BI projects:
A business analyst creates a draft BRD containing itemized business and functional requirements, each with a line-item number. This will be used to cross-reference items residing in other documents.
- The requirements line items will cross-reference to data contained in the logical data model.
- The data modeling tool will create denormalized structures to form a physical database design and the resulting table structures.
- The extract, transform and load specifications will reference the physical tables and map them to the ETL to load each table.
- Test case scripts are prepared cross-referencing each requirement to the logical model, physical tables and ETL mappings.
- The generated reports will cross-reference the requirements, tables and test cases for user acceptance testing.
As each of these main deliverables is stored in the toolsets repository - depending on the sophistication of the tool - table definitions, source-to-target mappings and other code may be generated to support the development lifecycle. In this manner, the toolsets establish a common way to create deliverables, provide a standard methodology flow and consistently apply checks to ensure that each deliverable is linked properly.
Requirements traceability is another traditionally labor-intensive (and, therefore, seldom-practiced) process that is nevertheless very useful to ensure that every requirement in the project has been supported by the data. There are two fundamental ways to effectively use requirements traceability:
Forward traceability: Every requirement flows into the next set of artifacts for design, implementation, test and deployment. This enhances BI development and highlights dependencies.
Backward: Every artifact item is traced back to prior artifacts, all the way to individual requirements. This ensures consistency from test, development, design and requirements. Backward traceability also proves useful for change control and impact analysis of proposed changes.
When these methods are applied, requirements traceability ensures that every articulated business requirement is being implemented and that every function delivered is supported. You also ensure that every requirement is tested to prevent production surprises. And you will be able to determine that no new extra functions are built that were not supported by a requirement. A sample flow is shown in the figure.
Requirements also need to be validated by other methods. Traditionally, the user reviews, comments and signs off documents. In order for this validation to be effective, the user should see the requirements both in visual and textual formats. For visualization purposes, data models and prototype screens illustrate the data structures. A walkthrough of the business questions and data model verifies that all the data exists in the model for the BI requirements. Prototyping tools can demonstrate the look and feel of BI screens and report contents. Once business users experience the look and feel of the new BI application, it becomes easier for them to understand the intended deliverable. Thus, new requirements software tools aid in user expectations management as well as streamlining development.
As with any technology, requirements automation software is a process enabler. Your BI team will still need the right expertise, clear roles and responsibilities, and a basic requirements-gathering process before successfully deploying these tools. Solid business/IT alignment goes a long way, too. Using these toolsets in client engagements has breathed new life into some moribund BI requirements conversations. And, as any experienced BI practitioner knows, if it helps engage business users, it must be worth trying.