I get many questions from clients interested in evaluating different in-memory technologies. My first advice is not to mix apples and oranges and clearly understand the differences between in-memory indexes, in-memory OLAP, in-memory ROLAP, in-memory spreadsheets and other approaches.
See more details in my recent blog entry "I forget: what's in-memory" to understand the differences. Then once you zero in on a particular segment, you can indeed do an apples to apples comparison.
Let's say we pick the category of in-memory associative indexes, which would include Microsoft PowerPivot, QlikTech and TIBCO Spotfire.
I recommend a 3-step approach to compare these three tools:
- First, compare all of the commodity features of the vendors and tools like data integration, portal integration, operational features like administration, security and others. You can leverage the detailed evaluation behind our slightly outdated 2008 BI Wave, if you are in a hurry, or wait for another month or so and the 2010 update will be published (it's in the last stages of editing at this point). Or if you are a Forrester IT - not a vendor - client, send me a note and I'll share a draft preview with you.
- Next I recommend using our BI consultants directory to figure out how many trained professionals for each tool are out there, so that you can feel comfortable that you can get help with the tools in your specific region and industry.
- Last but not least, evaluate features of these there vendors that are truly differentiated.
For example:
- Memory optimization. Compression ratios. What’s the theoretical and practical limit on the size of the in memory data models? What is the largest data model a vendor has in production? Since QlikTech has been in this game the longest, I typically hear from my clients that QlikTech has larger in production models than the others.
- Load time. How long does it take to load the model into memory? Load speed per 1Gb, for example. This'll largely depend on your particular hardware and infrastructure so you need to run your own tests.
- Memory swapping. What is the vendor approach for handling models that are larger than what fits into a single memory space? Microsoft and QlikTech in-memory models are limited to what'll fit into memory at one time. TIBCO Spotfire, however, can swap parts of the model in and out of memory, so you can indeed work with models that are larger than what would fit into memory at one time.
- Incremental updates. What is the vendor approach for delta, incremental, etc updates to the in memory model without rebuilding , reloading the whole model? Here, only QlikTech has a solution and can update its model row by row.
- Thick and thin clients. What are the differences between the thick and thin client versions in terms of functionality, interactivity, etc? What functionality do you lose when you run the in-memory model on a server and access via browser?
- Access by third party tools. Can you access the in-memory model via industry standard SQL and MDX reporting and analysis tools? Only Microsoft will let you do that by publishing PowerPivot models to SharePoint and turning them into a SSAS cube.
- GUI. Microsoft users will find a familiar Excel-like interface, while QlikView and TIBCO Spotfire UI is more visual.
- Advanced analytics. Are you interested in combining advanced analytics (statistical analysis, predictive modeling, etc) with your traditional BI reporting and analysis? QlikTech does not offer that capability. PowerPivot will leverage a few basic statistical and data mining functions that come with Excel and SQLServer. TIBCO Spotfire does offer advanced analytics functionality, S+ technology acquired from Insightful in 2008.
Did I miss anything?
This comment is Cross-posted from http://blogs.forrester.com/boris_evelson/10-09-01-not_all_in_memory_analytics_tools_are_created_equal#comment-6897