What aspects do I need to consider for an ETL tool for a strategic data warehousing development project in banking domain? The existing old warehouse is 3+Terabyte size and it is increasing on average rate of 17.5 percent per year.
Sid Adelman's Answer: There are some specific features you want in the ETL tool and they are:
- Performance - The ETL should be able to perform the entire ETL process well within your processing window. Since data warehouses tend to grow, it would make sense to benchmark this performance with about three times your data volume to verify meeting your window requirements.
- Function and ease - The folks creating the ETL process should feel very comfortable with the capability of the ETL tool to develop and maintain complex transformations.
- Data cleansing - The ETL tools all have some "lite" cleansing. Try out these functions on some of your ugly dirty data.
- Meta data - Some organizations use the meta data capability of the ETL tool as their primary meta data repository. If that's what you plan to do, take a close look at the meta data capability and verify if it will support all or most of your meta data needs.
There are other things to consider. For example:
- Does the product run on your chosen platform? The platform would include the operating system and the DBMS.
- How well does the product work with other products you have already chosen?
- Is the vendor financially stable? If the vendor is publicly traded, the annual and quarterly reports are readily available. If the vendor is not publicly traded, they should provide you with financial statements.
- What is the cost of the product? Include maintenance and required consulting and total cost of ownership
- Does the software allow customization that is adequate enough to support the way you do business?
- What is the administrative effort to support the product (number of FTEs)?
- What is the learning curve? How long will it take for your personnel to effectively use the tool?
- Issues involving recruiting or training staff. If you will be hiring to support the tool, research the availability of the pool of skills. Research the recommended classes, both from the vendor and from third party sources who teach the requisite skills.
- How good is the vendor support? Does the vendor have the capability to remotely diagnose problems?
- Is the vendor training adequate?
- The product (not the vendor) has what percentage of the market.
- What diagnostic tools come with the software? How does the vendor track product defects (bugs) and what is the status of those defects?
Chuck Kelley's Answer: The big question is can the ETL tool do the transformations you need to have done. Take a look at what you need to do and see how the tool accomplishes it. Do you get a file of 10 million rows every month and have to subtract current row from last month's row. Ask the tool how they would handle it. If it makes since, then great. If not, run!
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access