To read the first part of this blog, click here.

While current big data technologies help perform regular data mining on a much bigger scale, that’s only the beginning. Technology companies have for years been trying to make computer systems work beyond fetching results.  They’re venturing into the fuzzy world of decision-making via artificial intelligence. With the advent of big data technology and tools, a branch of AI called machine learning has greatly advanced. 

Machine learning deals with making computer systems constantly learn from data to progressively make better, more intelligent decisions. Once a machine-learning system has been trained to use specific pattern-analyzing models, it starts to learn from the data and works to identify trends and patterns that have led to specific decisions in the past. Naturally, when more data — along all of big data’s axes — is provided, the system has a much better chance to learn more, make smarter decisions and avoid the need for manual intervention and decision-making.

The insurance and financial industries pioneered the commercial application of machine learning techniques by creating computational models for risk analysis and premium calculation. They can predict risks and understand the creditworthiness of a customer by analyzing their past data.

While traditional systems dealt with tens of thousands of data records and took days to crunch through a handful of parameters to analyze risks using, for example, a modified Gaussian copula, the same is now possible in a matter of hours with two major differences: First, all available data can be analyzed and, second, risk parameters are unlimited.

Predictive Analytics

Machine language technology can use traditional and new data streams to analyze trends and help build models that predict patterns and events with increased accuracy and convert these predictions into opportunities.

Traditional systems generally helped identify reasons for consistent patterns. For example, when analysis of decades of data exposes a consistent trend, such as an increase in accident reporting during specific periods of the year, results indicated climatic or social causes such as holidays.

With big data and machine learning, predictive analytics now helps create predictions for detecting payment fraud, claims fraud, claims reporting volumes and trends, medical diagnosis for the health insurance industry, new business opportunities and opportunities for improved office efficiency. The industry can take advantage of these trends and turn them into opportunities for business growth and expense reduction.

Fraud Detection

The insurance industry has always been working to devise new ways to detect fraud. With big data, it’s now possible to look for fraud detection patterns across multiple aspects of the business, including claims, payments and provider shopping, and detect them fairly quickly.

Machine learning systems can now identify new models and patterns of fraud that previously required manual detection. Fraud detection algorithms have improved tremendously with the power of machine learning. Consequently, near real-time detection and alerting is now possible with big data. This trend promises to only keep getting better.

Organizational Efficiency

Automation of manual processes results in significant savings. But in huge, complex organizations, there are almost always overlapping or multiple instances of similar systems and processes that result in redundancy and increased cost of maintenance. Similarly, inadequate and inefficient systems require manual intervention resulting in bottlenecks, inflated completion times and, most importantly, increased cost.

Using data from internal systems, machine learning systems can study critical usage information of various tools, and analyze productivity, throughput and turnaround times across a variety of parameters. This can help managers understand inadequacies of existing systems and identify redundancy.

The same data sources also are used to predict higher and leaner load times. As a result, infrastructure teams can now plan to provide the appropriate computing resources during critical events. These measures add up quickly, resulting in significant cost savings and improved office efficiency.

The use cases discussed above are just the tip of the iceberg. Medical insurance companies benefit from better medical diagnosis for patients from claims, tests and prescription information. The entire insurance industry can achieve precise and targeted marketing of products based on past history, preferences and social data from customers and competitors.

It would be safe to say that big data is here to stay. No piece of data, regardless of form, source or size, is insignificant. With big data and machine learning tools and algorithms, combined with the limitless power of the cloud computing platform, the possibilities are endless.

Chander Ramamurthy is an architect with technology company X by 2.

Readers are encouraged to respond to Chander by using the “Add Your Comments” box below. He can also be reached at

This blog was written for Insurance Networking News. Published with permission.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access