The new trend has been demanding real-time data from the data warehouse: 1) to generate operational report at high frequency 2) to provide a golden source (real-time reference data) for multiple OLTP Applications. Which is the right ETL tool to extract data from multiple OLTP sources with no latency and importantly no performance impact on OLTP. RDBMSs are Informix, Oracle and DB2 UDB?
Sid Adelman's Answer: To those of you who believe that the support garment industry can produce pantyhose in a one-size-fits-all package, a real-time data warehouse might be appealing but you will deliver a compromise that is sometimes baggy and sometimes stretched to the breaking point. Analysts and knowledge workers need a stable source of data. When they run a query and run it again 10 minutes later, they want the results to be the same. Go back to the basics and learn the purpose of a data warehouse and don't let the current trend push you into a foolish implementation.
Chuck Kelley's Answer:In a word, "never." There will always be some degradation or latency. But most likely both will occur.
Les Barbusinski's Answer: Both Informatica and Ascential offer real-time/zero latency ETL engines that work with a variety of relational databases and packaged applications (e.g., ERP, SCM, CRM, etc.). How much of an impact these have on the OLTP environment depends on the implementation.
The only other near real-time ETL option I've seen is to use database replication to create a copy of the operational database on a separate server, and have an ETL tool monitor that replicated copy for changes that need to be posted to the data warehouse. Obviously, this method introduces latency factors that can make the data several minutes old before it gets into the data warehouse, but it has the benefit of having near-zero impact on the OLTP environment. Hope this helps.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access