How do you know a person well enough to understand their motives, their actions, their aspirations and needs? It can be difficult even among friends and relatives one has known for years, let alone for a financial institution with hundreds of thousands of customers. Some people say it's impossible for a large bank like Citi or Bank of America to really know its customers.

Yet the amount of publicly available data about each and every one of us grows all the time, to the point where an organization with the will, the patience and the right software can analyze in detail our financial transactions, our habits, our political leanings, our preferences and our geographic location, among other things.

Banks have been analyzing their customer data for decades, most thoroughly in the credit card business, searching for signs of fraud, willingness to upgrade to a new product or propensity to leave for a competitor. Today, many banks have projects under way to pull together customer data from all channels - branch, ATM, online banking, mobile banking, call center, social media sites - in one place, to mine that data in real time and use it to cross-sell, up-sell, detect fraud and keep customers in the right products. There are six trends guiding such projects.

1. The Big Data myth. The trendy phrase "Big Data" refers to data sets that have grown so large and complex that they become awkward to work with using standard database management tools.

Data volumes undoubtedly increase all the time. IBM estimates 2.5 quintillion bytes of data are created every day from a variety of sources including sensors, social media, and mobile devices around the world. IDC estimates the market for "big data" technology and services will grow at an annual rate of nearly 40 percent to reach $16.9 billion by 2015.

One bank customer recently described banks' data challenge to Boxley Llewellyn, global retail banking director at IBM, as "being in a big room full of data that's a little dark, so sometimes data gets trapped in a corner and sometimes it can't be found quickly enough. A wind of streaming data, social data and unstructured data is knocking at the door, and we're starting to let it in. It's a scary place at the moment."

But the idea that businesses need to store, mine and analyze every scrap of the customer data they collect is not practical.

"A lot of times when analytics and engineering people ask the business people what data they want, they get this answer back: collect everything and we'll sort it out on the back end," says Joseph Stanhope, senior analyst at Forrester Research. "That's not a data management strategy. There is too much data from too many sources coming at us too quickly for us to just save everything forever. You do need to be discerning about what data the business uses, which data goes to a KPI that shows us if we're moving the business forward. If people can't articulate what they need up front, they're not going to pick it up on the back end."

Gaming companies, for instance, don't mindlessly store all the data they collect on gamers, he observes. They curate the data to understand what is useful and what isn't, and create data hierarchies, schemas and categories to manage, condense, add and change information. "To understand this is more than technology, it's about people in the organization and the culture," Stanhope says. "If you can't evolve and change what you curate, then you do have to store and collect everything and business passes you by."

Edgar Enciso, executive vice president and director of customer intelligence at BBVA Compass, concurs. "We have a lot of noise around Big Data," Enciso observes. "The first challenge is to clean that information and define what data and analytic makes sense. We have information for everything and for everyone. However, when you try to be hands-on with the data, we have to clean it up and put it in a meaningful way so we can make the right decisions."

2. The use of predictive models to make better offers to customers." Once banks get that full picture of customers, they interrogate all the data they have and build predictive models," says David Wallace, global industry marketing manager, financial services at SAS. "They match the predicted behavior with campaigns for new or enhanced products, cross sell and up sell. They identify customers at risk of attrition and put programs in place to try and save the customer relationship because it's less expensive to keep customers than to get more. Predictive modeling is at the heart of all those activities."

BBVA is a case in point. It has three main goals for its customer analytics efforts, much of which are carried out in SAS Enterprise Miner analytics software, according to Enciso.

First, the bank is trying to make the right decisions to target the right offers to the right customers, through customer segmentation. The bank segments customers into the categories of wealth management, commercial banking, retail consumer and small business. It also performs lifecycle segmentation, grouping customers according to life stages, such as singles, independent professionals, young families and retirees.

The second goal is to understand customer profitability. "On the customer side and on our side, we want to have the right rates and pricing," Enciso says. "When we find customers who are not profitable, we try to find a way to serve them better, to keep them but put them in the right products."

The third objective is to analyze customers' life events and predict their future needs. "We're trying to see what are the customers likely to buy, what's their next problem?" Enciso says. The bank analyzes patterns in transactions and balance levels. "When we see that our customers are lowering their business with us, we're trying to find a way to keep the business," he says. If, on the other hand, a customer is increasing his balances, the bank tries to move that person to a higher segment with a better service level.

As it pursues these goals, BBVA is working to deliver intelligence to staff behind the mobile, online and ATM channels and in the branches to turn such insights into action and sales. "We are committed to deliver that intelligence to every point in the bank that our customers touch, and trying to make a better decision in front of them," he says.

"When a transaction happens in a channel - mobile, ATM or online - we need to have intelligence in the back end, so the tellers can react to that," Enciso says. One customer analytics-channel tie-in the bank is working on is through SMS messages: when a customer makes a purchase at the point of sale and he's near his credit limit, the bank will send a text message asking if he wants to increase his credit line. "If he answers yes, we increase that and he can continue shopping with a credit card," he says.

3. The 360-degree view of the customer is still a work in progress. They've talked about it for years, but how many banks truly have a 360-degree view of the customer, with a real-time window into their deposits, loans, mobile banking activity, online banking activity, ATM withdrawals, and teller transactions? "Getting a true 360-degree view of the customer is still very important, it never goes away as a critical issue," says S. Ramakrishnan, group vice president and general manager, Oracle Financial Services Analytical Applications. It's not just information about the customer that's needed, he points out, but value-added metrics and analytics around the customer's risk, profitability, preferences and how happy or unhappy he is with the company.

The 360-degree view can be used, for example, in pricing products and services for customers. "Loan pricing can be done more attractively if your prior report history is known," notes Ramakrishnan. "We can also bundle your relationship and price deposits intelligently."

Great Western Bank, which is owned by National Australia Bank and currently has $9 billion in assets, has been encouraged by its parent to grow in assets and revenue. "One of the challenges here is the bank is growing through acquisition as well as organic growth," says Ron Van Zanten, vice president of data quality. But until recently, measuring the profitability of products and customers has been impossible, as data has been stored in disparate acquired and legacy systems, including a Jack Henry core system, Clairmail mobile banking software, online banking and a Moody's business loan spreading tool.

Van Zanten and others spent six months identifying all the places the bank stored customer information. Then Van Zanten oversaw the deployment of Microsoft SQL Server 2012 to act as an enterprise-level data warehouse and business intelligence system.

To pull data from the data silos into the new database, the team built an ETL process with a standardized schema and used integration services from Microsoft's SQL Service Pak. Today, data from the core system, the credit card system, wealth management and cash management product all load into the central Microsoft SQL Server data warehouse.

Great Western, which is based in Sioux Falls, S.D., is also beefing up its data sources. Van Zanten is pushing an initiative to redo the bank's CRM application to capture more data about customers for the enterprise data warehouse and in the future, run predictive analytics.

He would like to have phone banking customer service reps and credit card customer service reps using the same CRM system as everyone else in the bank. "I want to be able to track every time the customer talks to us," Van Zanten says. "We know when they do their mobile banking and what they're doing." Reports alert staff to customer retention and cross-sell opportunities, for example if a customer has a couple of CDs coming due and they pay down their loans. "We need better resources for collecting customer behavior. Right now we have the web and mobile. I would like to know what's happening at the teller line, what's happening when they talk to the bankers, what customers say when we offer them products."

Analytics should help refine the cross-sell pitches. "We always have things going on, such as a credit card special or a home equity line of credit special," Van Zanten says. "We want to know why they say no, bring that back to the household information and compare it against people who have said yes. Our CRM system today doesn't track that well enough to get a good predictive model put together."

Among other data sources, the bank purchases external data such as home values. Expense data is drawn from the general ledger system. Budget and finance groups also weigh in on how expenses are allocated and a committee tries to make sure everybody (branches, lenders, etc.) gets a fair shake on expenses.

In mobile banking, the bank looks at the age groups of people who do mobile banking as opposed to online banking, to create profiles. "Mobile banking is probably a loser for us, but it is sticky," Van Zanten notes. "People stay with a bank if it's modern and convenient." Being able to cross-sell within mobile banking would increase the potential for profitability.

One of the advantages of creating one's own data analytics, Van Zanten notes, is the flexibility to change course and look at things different ways. So for instance, if a branch has $100 million in deposits and $80 million in loans, and gives the $20 million in excess deposits to corporate to lend out, the branch could be paid for that money or get nothing as a penalty for not lending all its money out. "You can do what-ifs and do things differently, whereas if it was a turnkey or canned system, they're going to do it according to industry best practice, but it will be one practice," Van Zanten says.

A primary goal for the new system is to be able to determine product, customer and branch profitability through analytics. "We want to be able to see what is working," Van Zanten says. For instance, when looking at branch profitability, "right now, some people think CDs are bad, because if we can't loan that money out, we can end up putting it in U.S. Treasuries that get .5% interest. But are CDs in general money losers? Where can we price ourselves from a loan perspective to be competitive and still make money? If we're making a lot of money on a commercial customer through cash management, corporate credit card, corporate checking account and other services, maybe it's worth a quarter of a point difference in rate and we could still get the same ROE from that person."

Being able to answer such questions and see a return on equity for every customer is part of achieving a 360 degree view of the customer. "We can make decisions that way so every deal is a win/win: the customer gets a little bit better rate because he has a great relationship with us," Van Zanten says. "We're getting income for services and our money is lent out in a safe way and a measured way."

The bank plays with data visualization, for instance, geocoding customers' addresses in the SQL Server so they can easily be turned into a map. This has proven useful as Great Western has found that some of its in-store Walmart branches are unprofitable and has consequently shut them down (it still has some branches in the megastores). "If you open a free checking account, you don't overdraft and that's the only product you have, it's really hard for us to make money off you," Van Zanten notes. "We closed a couple of branches, and we were able to run a query in a few minutes to find the nearest branch for any customer," Van Zanten.

A future data source to be fed into the bank's customer analytics engine is social media, which could provide further insight into customer profitability.

While watching a recent episode of 60 Minutes on addiction, Van Zanten was struck by the idea that in some cases, when people are addicted to food, drugs or alcohol, pheromones become active in the brain at just the sight of, say, chocolate or the McDonald's arches, and they get a warm feeling. "Wouldn't it be great to have your brand give that kind of feeling? Coca Cola may have that. Apple might be that way too, even when they roll out products that don't work, everybody forgives them."

4. Social media data adds a new dimension to customer analytics. To manage its online reputation, in 2009, BBVA began monitoring the web with IBM social media research software called Corporate Brand Reputation Analysis, as a pilot between IBM and the bank's Innovation department. During the first half of 2011, positive feedback about the company increased by more than one percent while negative feedback was reduced by 1.5 percent. Global monitoring improved, providing greater reliability when comparing results between branches and countries.

BBVA uses IBM Cognos Consumer Insight to identify the subject, type, date, author, title and country of online comments made about BBVA and its brands. News channels, blogs, forums, Facebook and Twitter are regularly checked and analyzed and reported on. The content of the comments is analyzed using custom Spanish and English dictionaries, in order to identify whether the sentiments expressed are positive or negative. The solution has been rolled out in Spain.

The bank can now respond to negative (or positive) brand perception by focusing its communication strategies on particular Internet sites, countering - or backing up - the most outspoken authors on Twitter, boards and blogs.

Following the deployment in Spain, BBVA will replicate the Cognos Consumer Insight solution in other countries, providing a single solution that will help to consolidate and reaffirm the bank's reputation management strategy.

"We're doing some tests, trying to send meaningful messages to the people who are interested in BBVA," Enciso says. "We're also trying to gather data in social media that's there to understand customers internally and externally, to profile the customers and send the right communication to them."

5. Crowdsourcing is informing product upgrades and new releases. "Some of the larger institutions have realized they can use analytics to learn about new lines of business and products, to ask customers what they think, and to get ideas, which is crowdsourcing," Llewellyn says. "Let's ask our customers, let's start to collect that data as well. And eventually that will go from the sentiment of the masses down to the sentiment of one, which says I can serve you uniquely."

When a company releases a new product that's causing problems, analyzing comments in social media sites or product review sites can enable it to quickly remediate, Stanhope points out. If, on the other hand, the new product is very popular, through analytics that can be quickly recognized and capitalized on. Even very small changes, such as moving a branch and ATM locator to the home screen of a mobile banking app, can sometimes make a difference.

6. Analytics software is becoming real-time, high-performance and more easily linked to immediate action. High-performance analytics software from companies like Oracle, SAS, SAP and IBM run on powerful hardware or clustered x86s and in some cases use in-memory databases to work with large sets of data with minimal time delays. "It's very beneficial, it lifts a lot of the traditional limitations we had on how much data you can collect and store," says Stanhope. "There's less preprocessing of data and fewer restrictions around the database schema." In-memory databases can better accommodate real-time transaction processing and help companies physically scale their technology environment to handle this kind of data. None of this is cheap, he observes.

Bank customers are interested in shorter life cycles and faster response times for predictive models, Wallace notes. "For large banks, the challenge is to execute on each one of those models and the resulting interactions much quicker and to get absolutely the best, most precise interaction possible," he says. "Those customers would like to develop more models quicker. That modeling process is not something that happens in hours today, it happens in days or weeks because the modelers have to wait for IT to provide the data set, explore the data set, come up some ideas, and test the ideas with different models, choose the right one, then put it into production. In some cases, that time period may be a number of weeks. Our customers are saying we need to cut that part of the model lifecycle down as much as possible. If we can do it in a day or two, that's the ultimate."

Customers also are interested in testing their models against all available data rather than a sample set. "Then your resulting prediction is going to be much closer to 100% correct," Wallace says. "Once that happens, then you link that with decision management software that makes the link between what you should do when the next interaction comes and instant action when that interaction happens." For instance, a customer who's looked at a particular mobile banking screen three times in the past hour may want an email with more information about that product.

"The result of that is you have highest opportunity to grow each customer relationship on the revenue side," Wallace says. "On the customer experience side, every time you get closer to delighting your customer by showing that you understand what their real needs are, without blindly sending them emails and credit card offers, it makes the customer view their institution as caring about them and understanding what their needs are."

This story originally appeared at Bank Technology News.