When is a hybrid approach right for the enterprise data warehouse?
After attending a recent analytics conference sponsored by Gartner Inc., I walked away with both new and updated information, as well as a lingering, disquieting observation. In a nutshell – there is lots of data out there with more pouring in every day. Yet it’s really hard to derive insights from it.
As someone once said “We have oceans of data, yet only puddles of information.”
While this is not breaking news, this perception persists largely because the prevailing enterprise data warehouse (EDW) architecture is unable to meet new, rapidly emerging business requirements. That means many new business requirements are depending on data that, in the best case, is at least being stored, but certainly not explored.
Fortunately, radical measures like rip and replace are unwarranted, as the situation can be addressed by augmenting the EDW. Augmentation implies adding a big data component, that is, a big data warehouse, or BDW, to supplement the enterprise data warehouse.
Adding a big data element to the configuration creates a hybrid solution that is flexible, enduring, and affordable. In other words, the fix is not binary, as the solution needs both architectures operating in pursuit of the same business objectives – getting information into the hands of knowledge workers quickly and easily.
Here are some broad scenarios in use today by numerous enterprises, and all made possible by a hybrid solution.
One of the first things that can be quickly realized with a hybrid architecture is self-service data access. Self-service is the ability for users to request datasets to query and generate either standard or custom reports on demand with only limited training and support from the IT team.
I am certain self-service will be an increasingly critical capability for organizations in the very near future. The fact is, our days are already defined by various electronic and/or digital DIY activities. So why not extend the reach to access and configure data in a way that best meets individual needs? After all, the person asking for information is the one that can best articulate the problem, making him more likely to find answers faster. And as data turns into information more quickly, the opportunities for generating business value increase.
It’s important to note that in this scenario, as well as most others, you need a BDW designed to do a lot of heavy lifting such as breaking down the data silos, handling all the complex data types, and sharing data responsibly. Only then can your preferred off the shelf tools take over to handle self-service from the datamart to the desktop.
Whether feeds from Twitter or data from Facebook and LinkedIn, you can learn a lot about your customer based on a user’s social media patterns. Popular themes correlate closely with personal interests, and both can be linked to create pretty accurate individual customer profiles. With the right tools, the abundance of available data can be mined to reveal almost everything an enterprise is eager to know: age range, gender preference, buying habits, physical activity, asset ownership, dining preferences, hobbies, political and religious leanings, and others.
Of course, enterprises aim to use data for good, not evil, where ‘good,’ for example, includes identifying ways to reward customer loyalty or incent a client in a meaningful way that was identified through an analysis of their public social media data. Examples include programs that offer free movie tickets, coupons for favorite fast food establishments, reduced rate fitness classes, transit passes, car wash discounts and the like.
Despite rumors to the contrary, the phone continues to be a significant communications tool. Large call centers field thousands of phone calls daily and also deal with voice mails and text messages. Whatever the medium, the enterprise first needs to categorize the communication (complaints, procedural queries, tech support, maybe even kudos) in order to route the message.
For example, a voice mail is first converted to text and then undergoes natural language processing in order to quickly identify just a few key words. Those words are clues that identify where the email should be forwarded for resolution. This automated flow improves response time, creates a better customer experience, and reduces both costs and customer churn.
A customer journey is simply a way of describing how your customers engage with your company. The journey could be an online experience, retail experience, a service-related activity, or some combination. It could the grand tour, or merely a day trip. Assume the former and imagine a customer that has some type of ‘account’ with you. And the practice is to ‘do something’ (renew a subscription, make a payment, update some kind of information, read this month’s rules and regulations about whatever) within that account on the 15th of every month. And last month that failed to happen.
What caused this and what happens next? Whether it’s a 404 error, or a failure, intentional or inadvertent, to renew a subscription, the enterprise has the capability to do something about it almost immediately assuring an improved customer experience.
Even leading edge healthcare applications continue to rely on the EDW for part of their function. Clinical structured data (tabular information) rests there, but images and diagnostics (unstructured data), for example, need to be stored in the big data warehouse. As extremely personal data is housed in both the EDW and the big data warehouse, the genomics application depends on the hybrid solution to ensure that sensitive, private information stays that way.
A hybrid approach guarantees that only the right data is provided to the right user at the right time. Permission levels can be based on parameters of your choosing and easily changed to reflect a different situation. As a matter of fact, all of the scenarios function with a built-in capability to protect privacy, manage data sharing, and ensure confidentiality.