In today’s big data world, analytics play a critical role in delivering actionable insights that empower personnel to act decisively and confidently in any situation. Many organizations are embracing analytics, making it a cornerstone capability of their strategies. However not all analytics solutions are equivalent or appropriate for specific needs. Given that the stakes are so high, making the best solution and vendor choice is paramount to the success of any analytics initiative.
Batch versus real-time; descriptive versus predictive; small data sets versus large data sets; analyzing geospatial versus time series or temporal data; self-service and on-demand access versus offline jobs – these are just a few of the topics that enterprises and vendors alike need to address. Given all these choices and more, confusion is prevalent in the market because analytics is a generic term whose meaning differs by vendor, use case and business requirement.
Determining the benefits expected from your analytics prior to evaluating different solutions is a recommended best practice that ensures the analytics you implement will best fit your needs.
This process starts with establishing a vision and specific goals that can then be mapped to the capabilities of your ultimate analytics solution. The vendor(s) that you choose to evaluate should transparently and thoroughly describe and demonstrate how their technology, products, and services meet your needs, goals and long-term vision. Making the wrong analytics software choice can not only be costly but also compromise your strategic and operational decision-making.
To aid you in your diligence, this paper identifies six key questions that you should address with prospective analytics vendors to ensure a successful outcome. Use this guidance to help ensure a best fit for your company and ongoing success of your analytics program.
Question 1: Can Your Analytics Handle the Complexity of My Data?
Success of any analytics program is of course highly dependent on access to data, so it is important to understand what data users require to make decisions. This data usually resides in a variety of different systems, formats and locations. This creates data and operational silos that force users into lengthy and manual data collection processes, and opens your business to errors in correlation and analysis of the data.
Users often resort to spreadsheets as a way of sharing data, exporting it from one system and importing it into another. This problem stems from the fact that the various analytics software operated in multiple departments or functional areas inherently limit what users are able to access. Even when enterprise data warehouses, data lakes and similar approaches are used to consolidate data, these approaches are costly and do not fully resolve all of the inefficiencies and problems caused by data silos.
It’s often not the case that the data doesn’t exist – it’s just that users can’t get to it. To divorce your organization from this data-rich, information-poor culture, look for software that has the adaptability and extensibility to span your existing systems. These include your data sources, your applications and even your analytics software itself. There’s no need to throw away what you have as it is possible to leverage the investments you’ve already made in an integrated solution going forward.
Let’s look at how analytics can help break down these silos and improve the quality of decisions made across your business.
Job one for any analytics software is the ability to access the data you need, including data that may not seem necessary. Some analytics software works only with traditional databases and structured data, while others are adept at also handling operational and streaming data and data in a wide range of formats. If you need to analyze time-series or spatial data, choose software that can not only access those, but also integrate them with other types of data in your infrastructure.
External data is often overlooked by analytics products. Data about wind and weather patterns, currency conversion rates, spot market prices, social media, traffic and demographics -- to name a few -- can be an invaluable part of your decision-making process.
Once you’ve identified the data you need, you should also assess when and how frequently you need the data to make effective decisions. If you are only performing descriptive analysis, then data may be collected and stored for a period of time before you make use of it.
But for real-time operations and IoT applications, analysis of streaming data becomes critical, and software that is capable of doing that should be identified. Even if you have been using analytics only periodically, consider what real-time (or near-real-time) analytics might do to operational efficiency.
The frequency of access and the currency of data are important since basing time-critical decisions on stale data will inevitably lead to poor operational performance.
Increasing volumes of data challenge the ability of analytics solutions to derive insights in a timely manner, if at all. Assessing the ability to process large volumes of data must therefore be among your evaluation criteria.
It is important to think about how analytics can help users understand large volumes of data. It is one thing to aggregate lots of data into monthly reports as this can be performed offline; it is another challenge entirely to analyze even greater volumes of data in real-time. Since humans cannot process so much data, look for solutions that can navigate through the mass of data to identify the nuggets of information that are important, be they anomalies in behavior or opportunities for performance improvement.
Where and how your data is stored is another important consideration. Most businesses use multiple different data repositories and frequently a mix of different storage technologies (e.g., SQL Server, Oracle, MySQL, MongoDB, Hadoop, etc.)
Data access challenges are exacerbated when repositories are in different locations as is often the case with operational silos, geographical silos, or both. This commonly occurs with global companies that have multiple autonomous divisions, and/or have grown by acquiring or merging with other companies.
You should therefore ascertain whether the analytics vendor offers interface connections to all the systems you need to access. You should also evaluate whether the vendor’s professional services team has experience integrating data from multiple sources and interfacing with multiple IT applications and technologies, or whether tools are offered for you to do the work yourself.
In summary, look for solutions that:
• Combine analytics results, information, and data from disparate sources including other applications into a single interface
• Enable you to create your own analytics algorithms and models as well as use algorithms and models from other sources
• Offer a library of analytics so you don’t have to reinvent the wheel
Question 2: Can Results be Visualized in Formats Relevant to My Business?
Consistently making confident data-driven decisions in a timely manner requires results and actionable information to be presented in a visually intuitive manner. Data, analysis results, and alerts must be displayed clearly and with contextual relevance.
Effective visualization is more than just the choice to display data in a chart versus a table. The efficacy of visualizing data and results must also include the ability to customize what is displayed and how it is displayed. Colors, shapes, icons, terminology, fonts and other visual attributes must conform to your business’ standards and common institutional practices.
Important insights and situations must be easy to identify at-a-glance to empower your staff to take immediate and appropriate action, especially for mission-critical operations and time-sensitive situations.
Effective visualizations bring forward insights, issues, challenges, and opportunities so that end-users can quickly recognize a situation and its severity along with relevant information about the cause and possible related issues. Combining results into a big picture view further enhances the ability of end-users to take decisive actions.
Your evaluation of any analytics solution should go beyond the basic question of “does your solution provide data and results visualizations?” and include the ability to:
• Display information in a web browser. This provides immediate access to data for anyone at any time and eliminates the need to install and maintain desktop and mobile applications to perform the same function. • Format how data itself is presented. The colors, shapes, iconography, terminology, and fonts all contribute to how information is perceived by users. These properties of the data should all be configurable based on data values, operating rules and/or conditions that might occur.
• Configure the presentation of data in ways that make end users highly productive. Most users have personal preferences and want to look at data in a format or layout that is comfortable or familiar to them. Layouts may also vary based on the use cases they are working on.
• View data in whatever format is most helpful in the decision-making process. Maps are frequently a good starting point for understanding where assets or issues are located, and these should be complemented by a wide range of charting types, data in tabular format, diagrams, documents, alerts, videos and access to third party applications where actions can be executed. All of these display formats should work in concert, providing the context users need to make informed decisions.
• Filter data using its attributes as well as by time, by spatial area or by network position. The ability to lasso regions on a map, for example, gives users the flexibility to hone in on dynamically-created areas of interest, as opposed to being constrained by pre-defined regions and time periods. This type of filtering also empowers users to quickly block out the irrelevant data as they work on problem-solving tasks.
Question 3: Does Your Solution Operationalize Analytics?
Democratization of data and the insights it provides enable your organization to scale and realize benefits across departments. Operationalizing and democratizing analytics helps make key information available when it’s needed most to whomever needs it.
Traditionally, advanced analytics has been performed as an offline process, with users requesting results and jobs run as a batch process to produce them. While this style of analysis is still needed for complex tasks, the pace of today’s business environment dictates that most answers to operational questions cannot tolerate delays of hours and days.
This approach also limits the utility of the analysis as first, the data is not that current when it is finally delivered, and second, users will receive all the data processed whether they want it or not, and then have to sort through it to find what they’re looking for.
End-users should have authenticated role-based permission to interactively analyze and explore data and the results from analytics. Interactivity means a few things.
First, it means that users have the ability to affect the analysis before it is performed, perhaps by providing input parameters that qualify what they’re looking for (for example, a specific time range, region, or range of values).
Second, users should be able to see the results in any appropriate format. While results are often presented in charts, in many cases seeing them on maps or in other visual formats may be more expedient in providing understanding. Third, users should be able to execute the analysis as frequently as needed. For example, they might be performing a what-if analysis and adjusting various parameters to determine their potential impact.
End-users should also be able to share the results they generate with colleagues and stakeholders to enhance collaboration and drive consistency throughout the company. This obviates the need for multiple people to perform the same analysis, improving productivity and freeing up IT personnel and resources for other tasks.
As described earlier, analysis of IoT and real-time streamed data requires a different approach. Operationalization in this context implies analysis of the data in-motion as it is streaming and providing insights to users proactively (as opposed to users clicking a button to perform the analysis). The ability to push data to a user’s screen can significantly improve their effectiveness as they will become aware of situations that require attention far sooner than before, and will eliminate time spent on tasks that may no longer be relevant or as important.
When choosing an analytics provider, it is important to have an understanding of your business requirements that will drive the determination of what you need analytics to deliver for your organization. It will also be imperative to ensure analytics solutions are able to deliver on the business benefits and results you are seeking, find analytics vendors that understand data quality and ensure analytics solutions can interface with data across your organization in terms of its disparities, volume and frequency of delivery.
Only consider analytics solutions that can present the data in intuitive visual formats that simplify the decision-making process and draw attention to important information and look for solutions that operationalize analytics allowing users to interact with analytical models, their outputs and the underlying data in different formats.
(About the author: Paul Hofmann is chief technology officer at Space-Time Insight)