The coming shortage of Analytic Skills

(Article also available here)

A   study by Avendus Capital published recently points out skill shortage in global analytics business.  According to this study, by 2018 the US will face a huge shortage of analytics professionals – a shortfall in the range of 140,000 to 190000 skilled professionals. The coming skill gap has serious consequences for

a)       companies where analytics plays a central role

b)      analytics service providers and

c)       analytics professionals  

Banking and financial services are the biggest users of analytics followed by retail, healthcare and pharmaceutical sectors in that order. Banking and financial services companies lead probably because they started using analytics long before analytics became a buzz word in the IT world. In fact many of the innovations – in data storage, business intelligence and deploying tools and business intelligence software (BI) for analytics - have been powered by demand and investments from this business sector.

While analytics is old game, the industry never attracted the kind of positive attention seen now. The analytics professional, sometimes referred to derisively as quant jock or what have you, never had it so good. Whatever the nomenclature, they essentially formed the bulwark of back office decision support, drawing useful insights from extant data. However, companies that integrated sophisticated analytics into their business decision process knew their importance all along. But in the recent recession, even as late as 2008 – 2009 or even later, they were laid off in droves from banks, retailers and pharmaceutical and other companies. They were on the chopping block whenever and wherever restructuring occurred. But today’s forecast shortage makes that look like ages ago. Clearly the analytics industry has turned around big time. 

It is common knowledge that the massive amounts of data now being generated from all-round in our digital world has engendered the big data genie. More data means more analytics which yields more insights. Suddenly companies have realized that they need a thought structure to handle this genie, else they risk losing their pecking order in their industry. It is this genie that has magically and swiftly created this colossal asymmetry in demand and supply of skillsets that we are currently witness to. 

In the short term – at least next 12 months or so, the demand for experienced analytic skills will outstrip supply. I know from my past experience in building and leading analytical teams that it takes approximately eighteen months to put together highly skilled analytical teams, albeit from ground up.

For companies that rely on analytics, the huge talent gap may force them to turn to third party analytics service providers. This may actually benefit some since if they move to offshore resources, their costs may actually come down. Consequently, these companies may discover new opportunities to expand their analytics infrastructure. On the other hand, companies that shy away from outsourcing, will have to freeze expansion plans or make do with the talent they have.

Major analytics vendors who already have built their teams will continue to stay ahead and will reap a bonanza for being at the right place at the right time. However, their time in El Dorado will be tantalizingly short since the demand for analytics has already spawned more players and most of them are investing heavily in nurturing good talent. I would definitely expect to see this intense competition put pressure on profitability for all the players. Of course, in the interim, I think there will poaching of talent galore!  I would not be surprised to see several mergers and acquisitions in this space. The big boys will swallow up the smaller players. 

There has never been a better time to be an analytics professional. The skill gap and poaching holds promises of greenbacks and an upward career graph.  This space is already attracting the younger and top talent every day which bodes well for the analytics industry.  As I have written earlier, I think the analytics industry is here to stay and will continue to play a pivotal role in constantly leap frogging the quality of decision making in the world of business in the foreseeable future.  

Think about it. The fortunes of the analytics business have been greatly impacted by happenings elsewhere – the genie sired by big data flow and data management technology. But I think in today’s business ecosystem where interconnectedness of data and insight is the key, it is hard to tell where one ends and where another begins.  But for all those who thought these guys were just quant jocks, wait till you see who laughs all the way to the bank. In the truest sense, it is the revenge of the nerds.