Don't Throw Hadoop at Every BI Challenge
The explosion of data and fast-changing customer needs have led many companies to a realization: They must constantly improve their capabilities, competencies, and culture in order to turn data into business value. But how do Business Intelligence (BI) professionals know whether they must modernize their platforms or whether their main challenges are mostly about culture, people, and processes?
"Our BI environment is only used for reporting — we need big data for analytics."
"Our data warehouse takes very long to build and update — we were told we can replace it with Hadoop."
These are just some of the conversations that Forrester clients initiate, believing they require a big data solution. But after a few probing questions, companies realize that they may need to upgrade their outdated BI platform, switch to a different database architecture, add extra nodes to their data warehouse (DW) servers, improve their data quality and data governance processes, or other commonsense solutions to their challenges, where new big data technologies may be one of the options, but not the only one, and sometimes not the best. Rather than incorrectly assuming that big data is the panacea for all issues associated with poorly architected and deployed BI environments, BI pros should follow the guidelines in the Forrester recent report to decide whether their BI environment needs a healthy dose of upgrades and process improvements or whether it requires different big data technologies. Here are some of the findings and recommendations from the full research report:
1) Hadoop won't solve your cultural challenges
For a couple of decades, technology professionals have architected and deployed BI based on strategy that often turned out too idealistic and impractical. By overemphasizing a single-enterprise BI platform, streamlined architecture, and a take-no-prisoners quest for a single version of the truth, tech pros have often been blind to the simple requirement of business users, who just want to get their jobs done. Such overzealousness breeds all types of disconnect between business and technology professionals. In Forrester's experience, most BI issues are cultural challenges that involve people and processes, not technology. Some examples (see more in the full report) of cultural challenges for which Hadoop certainly is not the solution include:
- Different points of view on who owns BI.
- Disconnect on BI goals and priorities.
2) A variety of new technologies and processes can close data-to-action gaps
Organizations can solve multiple data challenges with new and different technologies and processes; however, these don't necessarily have to be Hadoop-based platforms. For example, Forrester recommends that BI pros deploy (see more opportunties in the full report):
- Systems of Insight (SOI) for actionable insight.
- Agile BI platforms to improve self-sufficiency of business users.
- BI on BI to improve business and technology alignment.
3) Sometimes BI on Hadoop is indeed the right answer
Once BI pros filter out the noise of trying to solve nontechnical challenges with Hadoop-based big data technology and strengthening their BI environment with appropriate data technologies required by the business, they can proceed to considering Hadoop for the following BI use cases:
- Justify Hadoop investments by reducing budgets allocated to proprietary systems.
- Turn your Hadoop data hub into a sandbox for business analysis and data scientists.
- Extend Agile BI to big data with Hadoop on-demand data marts.
Alas, nothing in this world comes for free. While there are indeed multiple advantages to deploying BI on Hadoop such as lower costs, linear scalability, data exploration, and on-demand data schemas, BI pros deploying BI on Hadoop applications will need to deal with at least the following (and more) implications:
- Data in Hadoop data lakes will never be 100% clean and integrated.
- Data governance must adapt to different stages in the life cycle of big data.