Do I need a data warehouse to do data mining? My IT department moves at a snail's pace. By the time my unit sees a data warehouse delivered... anyway, my transactional database is ripe for data mining applications.


Sid Adelman’s Answer: No, you don’t absolutely need a data warehouse to begin data mining. But you do need clean, accurate data that is well understood and the creation of a data warehouse usually (and should) include cleaning up and documenting the data. Even if you pull the data from a clean data warehouse, you still have additional work to transform the data and get it ready for data mining.

Ross Armstrong’s Answer: You probably go to the restaurant and ask to see the dessert menu first. Get to the good stuff as soon as you can. You don’t need a warehouse first, but there are a number of considerations you should plan for. The process of preparing the data for mining is important. Remember, the mining algorithm analyzes the absolute values without consideration for context. If you have a column that changes valid values or changes meaning over time, then the results of the mining will be skewed unless you remove this data first. If there are spikes or errors in the data, then they have to be removed. Since the mining process might take a few weeks or months, it is necessary to not update the data for that period of time. This may not be practical in a transactional system. In short, you generally don’t need a data warehouse, but you’ll probably need something similar to put the data into to mine it.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access