Businesses are seeing a greater need today to enable fact-driven decision-making, forecasting, planning and inventory control. Request for proposals from customers seeking data warehousing, business intelligence and analytics solutions are drawing scores of responses from established, niche and boutique vendors of all sizes.

Each of these firms brings economies of scale, geographic location and partnerships with technology vendors as key differentiators within their proposal in an effort to win business from the customer. While these aspects certainly carry their own little merit to aid in selecting the software vendor, customers need a stronger criteria set that is relevant in order to evaluate these vendor proposals from common ground.

Irrespective of the size, complexity or scope of the DW/BI project, answers to the questions below serve as a good litmus test to narrow down a list of potential vendors.

1. Does the bid consider the end to end impact on the DW/BI platform in the proposed project approach? The answer to this will clearly show the depth the vendor has in executing turnkey DW/BI projects. It would also give the customer a clear indication of how well the vendor has understood the business problem and the value-add they can bring to the table.

2. Have the requirements been separated into source system changes, data integration changes and BI layer changes? A satisfactory answer to this question usually indicates that the vendor is rightfully dealing with the requirements provided, since the requirements have been well-dissected before delivering the proposal.

3. Were delivery dates independently planned or dependencies considered for each of them? If the effort for streams of work was estimated as a black box, but when the project was planned, dependencies were considered, the customer is in safe hands and will realistically see fewer surprises in the plan.

4. Has a system cutover plan been included with the proposal? Are the changes going to be done in an online or offline mode? Has downtime been accounted for in the deployment plan? Usually, inclusion of such activities in the plan and the proposal indicate that the customer is not a pure development shop alone, but will ensure that the project is in production before they deem the project complete. It shows that the vendor is willing to stand by what he or she develops and is invested in getting it right.

5. Has a rollback mechanism or running a parallel system been considered as part of the effort estimates? Not all projects go right at the first installation; failure-proof planning, if included in the proposal, shows the vendor is not afraid to face reality.

6. Has a “code freeze” time been mandated in the proposal to effectively push changes to production systems? How does the solution account for catching up with other development initiatives “in flight”? Quiz the vendor about this problem and how well “continuous integration” and “automated nightly builds” will help address this problem. Look to see if this is a part of their proposal.

7. Is there a mechanism to streamline and govern the use of new data available in the data warehouse? Has metadata been published on reporting/ad hoc query tools effectively, to be used by power users who need to use the data? Do the power users have access to a documented guideline to develop reports or query the newly published data and functionality?

8. Will the business glossary terms, their relation to columns in the database and report attributes including derivations, consolidations and aggregations be documented? Ensure the handoff includes a well-documented solution cookbook, run book and a meeting to pass the details on to people who will continue to support the system and functionality developed.

9. Is there a mechanism to point out which line of code, in the newly introduced functionality, and attributes to degradation in performance, if any? Ask for a functionality-code-performance co-relation report to ensure the vendor has mechanisms to point out the code that contributes to service level agreement lapses.

10. Does the proposed staffing model consist of BI personnel who understand data and data integration personnel who understand BI? Is the team led by a DW/BI-certified project manager/lead?

11. Can code quality be measured? Can some of the metrics (such as right first time; defect ratio per change or the quality index; performance metrics of the system; and coding metrics like review comments per change, code quality, documents for a change, etc.) be inferred from the metrics the vendor is capturing during the project lifecycle?

12. How does the vendor plan to measure productivity gains and efficiencies as his or her team gets familiar with your DW/BI environment? Are there any reusable tools, solutions or components that are considered to be built as a part of the bid, and who owns the IP rights for them?

13. How much of the testing can be automated? Typically, regression testing cycles on DW/BI projects cost more than the development effort. Having test automation and an automated test data creation process will ensure that effort is well-balanced, while reducing the total cost of the project and creating a repeatable automated test scenario.

14. How is knowledge transfer to a customer’s DW/BI team planned? This ensures that the vendor has thought about the whole project lifecycle, including transition of knowledge to the customer.

15. How much of production support and warranty is included in the proposal? This shows the vendor’s confidence in the proposed solution and how well they are prepared to support something they developed.

16. Is there an estimate view by functionality, project phases and by technology components? This gives the customer a chance to de-scope nicely, which allows for the inclusion of additional functionality and provides an opportunity to keep project costs low. This also gives the customer the chance to bring some phases of the project in-house, based on resources available. It helps the customer to cut costs, engage in-house resources effectively and keep a constant vigil on the vendor deliverables, thereby avoiding any big surprises.

17. Is there a relevant case study in the proposal that indicates the experience of having dealt with such projects before? Can references pertaining to the showcased case study be provided?

18. Are there any risk/reward clauses in the proposal? Inclusion of such a clause, usually when tied with a fixed bid made for the project, indicates that the vendor is very confident of executing it well.

19. Are any relevant proofs of concept that have been showcased on cutting-edge technology being offered in the proposal? Often, such proofs show that the vendor has a lab-type environment and an R&D division where the cutting-edge technology proposed has stood the trial of strength.

20. Lastly and most importantly, if the vendor takes the time to present what differentiates them from other vendors or what elements in the proposed solution are unique differentiators, the customer should have enough ammunition to rank vendors and award work to those who come out on top.

Ideally, this list of 20 questions will act as an acid test to rank top vendors who respond to a customer’s DW/BI RFP. A next step could be to pick the top two or three vendors, award them pilot projects in different streams and measure their system development lifecycle metrics at length. This will aid in the decision process of selecting a strategic DW/BI vendor partner.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access