Selecting software may be a chore, and the process can often delay a project. Reference checking is the least time-consuming way to learn how the product functions in the real world. Most organizations acknowledge that they will check references, but the reference-checking process is often disorganized and in many cases abandoned (sometimes because they already know which product they want).

Let's be clear; the selection of the right product and related utilities is a critical success factor for most projects. Without proper reference checking, your organization has not performed the due diligence required for such an important decision. There is an additional benefit to reference checking: if you pay attention and ask the right questions, you will gather lessons learned and best practices that go beyond what's the best software, and you will learn some optimal ways to install and use the product.

Selecting and Gathering References

Your first step is to contact the vendors whose software you are evaluating to request a list of references. If the vendors are smart and well organized, they will give you their best references (the ones who will say nice things about them and their products). Therefore, recognize that you will be getting a somewhat biased view and one that is not a representative sample of their customer set. That's okay. Even with this select group, by asking the right questions, you will be able to discover the dark side of the vendors and their products. Some vendors want to participate in the call. This is a bad idea. If they participate or even just listen to the discussion, the responses from the references will be much less candid. Decline this arrangement.

Desired Types of References

If you don't tell the vendors the types of references you want, they are likely to give you some that are of no value. Tell them what you want and what you do not want. The following are some suggestions for the types of references you want to call.

  1. The reference site does not have to be local (you will be using the phone, not visiting).
  2. The reference does not have to be in your industry, although being in your industry will enhance the credibility of that reference to those reading your report - you will create a report that details the questions asked and the reference's responses. On the other hand, if the reference is in your industry and if the reference identifies you as a competitor, they will not be as forthcoming with information - they may not even want to talk with you.
  3. The reference should have been actively and productively using the product under consideration for at least six months. You don't want anyone in the throes of implementation, and you don't want anyone who is just playing with the product.
  4. You want the references to be on the same platform (operating system) that you intend to use. The vendors should not give you a Windows reference if you plan to be on a version of UNIX.
  5. You want a reference that has at least as many users as you intend to have and at least your expected transaction level, degree of complexity and usage per user. You also want a reference that has a database of at least the size you are planning. Note: This is to verify performance, not function or ease of use, so you will want at least one in this category.
  6. You don't want a reference that has a financial or marketing association with the vendor's organization - they may be biased.
  7. If possible, ask for references that have the same products you have already chosen. For example, if you are a Business Objects shop, you would ask the vendors for references that are also using Business Objects. If you have PeopleSoft, ask for references that have PeopleSoft installed and actively running.

Next, you want to find other references that are using the software. If possible, connect with user groups. Attend their meetings and talk to some of the more vocal members at the break and after the meeting. Many of these products also have local user groups. Tell the existing software users that you are evaluating the product and ask for their time to answer a few questions. You can interview them right there or sometime after the meeting.
Finally, ask around and locate other companies using the software. It's especially useful to talk to installations with more than one product in the category you are evaluating as well as one with a competing product. These organizations are in an especially good position to tell you how the products compare.

Part 2 will continue the discussion about the process of checking references and will include some questions you want to be asking.  

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access