Organizations are collecting increasing amounts of disparate data, sometimes more than they are able to handle. They realize, however, that analyzing this data can provide competitive advantage. Big data continues to be an active area of interest for many organizations across various industries, but numerous questions still exist regarding big data initiatives. How mature are these programs? How far have the companies advanced in their big data initiatives? What does it take to succeed? Big data maturity is an evolution of an organization to integrate, manage and leverage all relevant internal and external data sources. It’s a journey that involves technology, processes and - very importantly - people.
In order to help organizations understand where they are in terms of their big data efforts, TDWI launched its Big Data Maturity Model in late 2013. This model was developed as a result of questions from our members about what they should be doing (if anything) big data and where to start. They want to understand what other companies are doing with big data, and the model allows organizations to see where they stand relative to their peers in terms of their big data efforts. The model consists of five different phases of maturity: nascent, pre-adoption, early adoption, enterprise adoption, and mature/visionary. It also contains a chasm. Additionally, the model includes an interactive assessment that can help to guide business professionals in their big data efforts.
The big data assessment consists of approximately 75 questions across five different dimensions. These dimensions are:
- Organization: To what extent do organizational strategy, culture, leadership and funding support a successful big data program?
- Infrastructure: How advanced and coherent is the architecture in support of a big data initiative?
- Data management: How extensive are the sources of data for big data analytics? How is this data managed?
- Analytics: How advanced is the company in its use of big data analytics? What kinds of analytics are they using?
- Governance: How coherent is the company’s data governance strategy in support of its big data program?
At the end of the assessment, respondents receive their overall score as well as their score relative to their peers by both industry and company size. They also receive feedback regarding expectations for the phase and some best practices companies are using to move to the next phase of the big data journey.
As of January 2014, about 300 companies had taken the assessment. On average, the scores put respondents in the pre-adoption/early adoption phase of the model. Broadly speaking, this means that companies that are undertaking big data projects are generally in these initial phases. A number of companies that took the assessment rated in other phases, either earlier or later, in their stages of development.
On average, companies received some of the highest scores in the organizational dimension. In particular, respondents, regardless of company size, scored higher on “fuzzier” questions in this category, such as innovation being valued at their company or that their company had a collaborative culture. In general, they scored lower around concrete actions for big data, such as obtaining business funding or developing a strategy or roadmap. In other words, while many respondents believed that cultural readiness for big data may exist, they were still fairly early in terms of any business funding or overall strategy. Additionally, while some of the respondents also had at least one executive sponsor in place, early in a big data journey this sponsor is typically in the CIO organization - not in the business.
Case in Point: an Example
In considering the various phases of big data maturity, it is helpful to look at an example of one company’s approach to solving its organizational big data challenges. The company is a financial and insurance services company located in the U.S that started its big data journey in 2010 with a business goal to provide world-class customer service. This company has been in business since the 1900s, serving the finance and insurance needs of its customers to their satisfaction and enjoying loyalty in return. However, in the early 2000s the marketplace evolved and became very competitive. The company started to see customers migrating away from its services, especially its second or third-generation customers. The CMO and CFO anticipated the problem and developed a new set of strategies to catapult the firm to a leadership role and empower its employees and its customers during interactions with the company. They engaged well-known consultants for this exercise and started to lay out a plan.
The promise of the new platform to customers was its availability anywhere and anytime in the world in a secure environment. The internal goal of the platform was to create an information mashup that could assist call center employees who were working with a customer. The idea was the call center staff could discuss issues related not only to the current call but also market and cross-sell products or services depending on the level of interest during the call by the customer. Data for the project came from multiple business units as well as external sources such as social media and emails. The strategy required not only alignment of data and analytics; it meant adoption of the platform by different teams across the company.
This company had a head start in that business leaders were working together to understand complex issues and why they were unable to solve them as a team. In our Big Data Maturity Model, we expect this behavior around the “Early Adoption” maturity stage, and organizations typically have a road to pave to get to this level of maturity.
An important factor to consider is the size of the organization and its activity focus, which can create complexities that need to be understood from a process perspective to create the right organization roadmap. In this case, the organization spent 24 months on this project. The leadership teams (along with employees and consultants) put the following organizational pieces in place.
Mission Statement An organizational mission statement was introduced to emphasize the importance of the program and its success.
Organizational Alignment The enterprise had more than 15 business units, each with its own operational systems, data warehouses and master data. Additionally, each unit managed its own metadata. The new system needed to integrate data from all of these systems into one corporate data warehouse and CRM application that would be used to feed the new call center application. The organization took the following steps to align the business units:
- Success metrics: A very interesting piece of the plan was tying success to adoption of the platform by employees internally and customers externally.
- Collaboration: Sharing of data and KPIs was necessary to create an integrated platform. At the same time, the ability for data and business analysts to collaborate on news articles, analyst reports, financial and insurance compliance data and competitive data was vital to ensure a robust foundational platform for decision support and analytics. The company launched a collaboration portal that employed gamification strategies to create an easy and dynamic platform for making decisions and a focus on news and its impact. Internet search and data collection were automated within the platform. The data collected consisted of a large set of patterns and verbiage from the Internet, which was then added to the list of news items to be considered for risk and profit management by the different teams. Once the data was collected, the list was updated and a new email was sent to each user who needed to look at the data and provide a gamification-based yes/no and priority rank for the item if marked yes. By creating this type of collaboration, the enterprise was able to increase the productivity of the teams, as they could get data automatically rather than search for it manually, which could take hours. Another benefit was the collaboration among various users and their perspective on the data and its priority to the organization, which helped the governance process evolve and increase adoption with user alignment. Competitive research is one area in any organization that can benefit from such an exercise.
- Governance: While governance is called out separately in the Big Data Maturity Model, it is important to highlight here. A planned data and program governance program was launched with a mission to provide a verified and trusted platform. The governance model included stakeholder, owner and business user roles that were defined, and each committee was empowered to make decisions for the program. Data privacy and compliance was also handled as part of the governance process. The governance board was created at different tiers in the organization: the executive committee was selected directly from the CEO and CTO offices, the program steering committee was nominated by the executive teams and the execution teams were nominated by the program steering committees. In recent years some of the key offices have been managed with election of the members to the office.
Skills and Training A key focus area identified by the executive teams was the newness of some skills required for the effort including social software, voice analytics and integration of existing skills like building advanced models and analytics. The company formulated a strategy to train more than 500 users across the business units. Some of the training included soft skills and communication skills. Another facet of the training was to learn the use of the new platform along with transparent sharing of the same to a customer, if the customer wanted to see the data. Multiple parts of the system necessitated training for various people and teams:
- Call center teams needed training on the overall system, screens, sharing, sentiment analytics and capture of data on customer interactions.
- Data science teams were trained to look at segmentation and explore behavioral analytics on a statistical platform.
- Business analysts were trained on executing behavior and response algorithms in a multidimensional population across multiple geographies, demographics and economic conditions, where they could monitor the social media trends.
- Bank and mortgage analysts worked on their proprietary algorithmic platforms using data from the new system, where they were able to create more effective campaigns.
Customer Alignment The final piece included how to introduce customers to the new program and coaching them about the advantages of the platform. Numerous approaches were rolled out, however, the most successful approach was communication and collaboration utilizing Android and iOS platforms via tablets and smartphones. The customers were invited to become users of the new platform by executives across the different business units. Additionally, call center staff members were assigned as “digital personal assistants” to the customers when they interacted on a call or email with the call center. The customers were allowed transparent access to their data as seen by the call center teams, which provided them a high amount of confidence in the discussion and the process. Customers could ask for a manager or supervisor to participate on a chat and they could rank and rate each interaction.
Preparation and alignment for the project resulted in strong adoption of the platform. The platform provided an opportunity to reduce costs by aligning data storage and collaboration, and most importantly created a new cross-sell atmosphere that resulted in excellent new customer and current customer participation. Ultimately, the organization felt involved as part of the process. The company has since moved into aligning visualization tools and has executed mobile data integration and management on the same platform. The key was putting the organizational pieces in place to make this possible.
Of course, organization is just one aspect of big data maturity, but it is a very important one. Participation in the big data assessment can reveal more about the other dimensions of big data maturity.
Respondents can take the assessment at the TDWI site at tdwi.org/BDMM.