In the era of big data, much has been written about how difficult it is to find analytics pros, otherwise known as data scientists. A 2011 McKinsey study claimed that by 2018 the U.S. could face a shortage of 140,000 to 190,000 such professionals, as well as 1.5 million managers and analysts needed to analyze big data and make decisions based on their findings. 

This article assumes you’ve already been lucky or skilled enough to hire some of these rare people and covers how to keep them engaged and retained. Mu Sigma employs nearly 2,000 data scientists, so we have a lot of experience keeping them happy.

What’s the secret? Over the years, we’ve found three main success factors:

1.   Foster collaboration. Data scientists are essentially schooling fish; they need to bounce ideas off of like-minded people. If they work alone or in too small of a group, they get lonely and bored. Solitary confinement saps them of their natural curiosity and stifles their creativity. Brainstorming and collaboration are key to their survival and ability to flourish.

2.   Allow them to spend time on discovery-driven analytics, not just problem-driven analytics. Data scientists need the freedom to explore their ideas  and often the wackiest ideas lead to the best breakthroughs. Be sure to give them the opportunity to spend time on discovery-driven analytics, not just problem-driven analytics. What does this mean? Traditionally, analytics teams are problem-driven; they are asked to solve a particular problem by applying analytics techniques. For instance, “Why do we see a dip in sales?” or “Which insurance claims are likely to be fraudulent?” Increasingly, however, businesses are giving their data scientists the latitude to perform discovery-driven analytics, allowing them to explore data for patterns and see where it takes them.

You stand a much better chance of discovering a game-changing insight when you don’t start with preconceived notions. Does each path pay off? Of course not, but the ones that do tend to be more insightful and impactful.

A related approach we love is the concept of the Simmer Project. An approach introduced to us by Microsoft, Simmer Projects are essentially side projects that aren’t expected to contribute any revenue in the short term  and are really just experiments. They simmer on the side for a few months – or maybe even a few years – while their creators watch and wait, occasionally adding ingredients. Then they  decide if the projects should be moved to the front burner or scrapped.

Each researcher typically has one or two Simmer Projects going on at any given time, and the best ones eventually get put on Microsoft’s roadmap. I’m told that some of Microsoft’s most well-known products have begun in this manner.

A great example of a company taking a long-term view on innovation – instead of putting every available body to work on short-term goals –  Microsoft has given its employees creative license to experiment in the kitchen.

3.   Ongoing education. The field of analytics is constantly changing. New technologies and approaches are emerging every month. Just like software developers, data scientists need to be up to speed on the latest tools and trends. It’s crucial that employers give them the opportunity for continuous learning and development – from mentors, from each other and through more formal training, education and conferences. They like the continual challenge, and, frankly, most data scientists I know (myself included) love to show how smart they are. 

Our organization runs an internal training program called Mu Sigma University that not only exposes new hires to our methods, but also serves as a platform of continuous learning for veterans. It’s not just a benefit to employees, but ultimately to our clients, who have access to extremely well-trained analytics consultants and practitioners.

These tactics have all worked for us. What steps is your organization taking to preserve and retain these precious resources?

Image used with permission from Thinkstock.