Author's note: David wishes to thank Mike Reed, principal at EWSolutions, for his invaluable contribution to this month's column.
During the process of product selection, a great deal of attention is given to the functional capabilities of the software being evaluated. While this aspect is obviously important, ignoring the technical mechanisms by which the software actually operates can be fatal to a project.
In order to evaluate the functions/features of products being considered, most corporations and government entities understand that detailed scripted scenarios should be performed with software vendors on the short list of candidates. Unfortunately, many companies neglect to investigate the technological underpinnings of these products during the evaluation phase. This can lead to the purchase of a functionally excellent product, but one that is difficult (or impossible) to support in the customer's environment.
It must be stressed to the vendor that this phase of the software selection process is designed to evaluate technical issues (i.e., Does the server run on Windows?), not functional ones (i.e., How is payroll processed?). The vendor should be provided with a list of questions in advance and given adequate time to prepare a response. Someone in the organization should be designated to obtain clarifications for the vendor where required, before the on-site meeting.
Major Evaluation Criteria
Product architecture: Examples include processing through application tiers, middleware and messaging components, and scalability.
Data protection and restoration: Backup and restore methodologies, and archiving.
Security: User authentication and authorization, transaction and database security, and directory services support.
Tools: Diagnostics, session control and print spooling.
Performance: Performance degradation when multiple application windows are opened on the same workstation, thread control between tiers of the application, and network bandwidth and latency, etc.
Reporting: Support for ad hoc reporting, third-party integration and custom report design.
Support for client-developed applications: Tools and control mechanisms, configuration management and methodologies.
Vendor demonstration of selected features: The vendor should demonstrate the most important technical features (i.e., assignment of security to a user, monitoring application performance, etc.).
The most important criteria for the success of a technical architecture software evaluation are the presence and participation of individuals qualified to hold the discussion. Once the proper subject areas have been determined, it is imperative that the customer provide a "jury" of individuals in the proper disciplines to ensure that subjects are properly covered. These individuals must be tasked with the background work to understand what the vendor is proposing and fit that proposal into the "template" of their unique environment.
It may not be necessary to designate a single individual for each of the following areas. Some employees will be able to speak to multiple subjects. If both the customer and the vendor do not send qualified individuals to the meetings, this effort will be severely hampered.
Overall process administrator: An individual responsible for the overall coordination of the "jury." This individual tallies ratings from the other participants and collates a collective score for each function point. This individual is usually also responsible for reporting progress to, and getting clarifications from, higher levels of management within the company.
Network administrator: To address issues such as bandwidth requirements and network protocols.
Database administrator: To discuss needs for specific databases (e.g., product only runs on Microsoft SQL Server).
System administrator: For discussion of platforms, disk space usage, CPU requirements.
Security officer: To examine implications of LDAP (lightweight directory access protocol) support, single sign-on and role-based access.
Application programmer: To discuss availability of vendor-published API (application programming interface) to enable custom enhancements/extensions.
Web programmer: To investigate Web server and browser requirements, the need for active server pages or other software dependencies, etc.
Functional representative: To aid the group in understanding the objective.
With the proper preparation and engagement, technical architecture discussions can be extremely fruitful. Various architectural features of the products can be compared and contrasted. Some vendors have been surprised to see that customers are interested in how things work, but soon realize that this is to everyone's advantage. A full understanding by all parties will help ensure a smooth rollout, implementation and ongoing support of the product. A good "out of box" experience is best for all concerned.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access