Is There A Data Science Skills Gap or A Hiring Hot Mess?
I wrote this for Fast Company three years ago. This is what I saw, and still see, is holding back the hiring process across tech.
Vin Vashishta | Originally Published: June 17th, 2017
I feel for recruiters trying to fill data science and machine learning job openings. The list of requirements is pure bravado with a side of madness. 10 years of data science with 5+ in natural language processing and a Master’s/PhD paying what a software developer makes. Never mind that I can count on my hand the number of data scientists who were building for production in 2007. Others ask for 3 different programming languages, 10 platforms, niche algorithm set, leadership skills, and I am only halfway through the job description.
Ask any tech recruiter and they will tell you about the stack of these job openings that have been sitting on their “must fill” list for the last 6 months to a year. Every 2 weeks, the client calls and berates them for not being able to provide quality candidates. After a while everyone involved throws up their hands and calls it a skills gap.
Dennis Miller used to say, “Now I don’t want to get off on a rant here…” and neither do I but we must face some painful realities on the road to solving the problem. Companies that do not get the data science and machine learning talent they need will not be in business in 5 years. Companies have a choice; either be Amazon or Sears. The right talent is a big part of that distinction.
Fixing the Hot Mess
Google does not require a PhD to be a machine learning engineer. A recent survey found that only 1 in 4 data scientists have a PhD. Yet I still see this advanced degree as a requirement on data science and machine learning job descriptions. Toss it unless you are investing heavily into advanced research.
The years of experience insanity needs to end. Forget years and start thinking in terms of problem-solving abilities. I love formulas so here is mine for hiring a data scientist. Platforms + Business Problems = Required Skills. No years of experience in the equation. Has the candidate solved the business problem on the same/similar platform before? Data scientists are used to working with uncertainty. We are used to turning business problems into technical solutions. Tell us your problems, show us your platforms, and take us to your data. We will outline a roadmap in the job interview that leads to a solution. If you like it, hire us.
Both these mistakes, and several of the other common ones, come from the myth that every data science team member needs to be a Unicorn. Most businesses need 1 person with the rare trio of strategy, engineering, and mathematical modeling. The rest of the team is built around this person and in support of this person. Sensible requirements have a lot to do with the right team structure. When every member is not expected to do everything, hiring gets a lot easier.
There’s 1000 Yous, There’s Only 1 of Me
When candidates start sounding like Kane, you know demand is high. For anyone with “data scientist” or “machine learning” in their job title, things have gotten a bit strange. We are all trying to stay humble and grounded amidst a massive hype cycle that we neither started nor perpetuate.
This is the other side of the hot mess. The hiring practices need to adapt to the reality we are in now when it comes to data science and machine learning talent. No company likes the thought of chasing after a candidate. When I was hiring software developers a decade ago, we tossed aside candidates who felt they were a hot commodity calling them aloof diva types.
The reality is businesses need a process that is streamlined to attract rare talent. Data scientists and machine learning practitioners now fall into this category. The businesses having success finding data scientists are recruiting for these roles in the same way they recruit senior executive talent. It is a relationship building process. It is more focused on the company selling the position than the candidate selling them self.
What Does “Better” Look Like?
The bigger problem is that many companies view data science and machine learning as check boxes on a to do list. Hiring a data scientist checks the box and they are done. Businesses that do not have a connection between the role and ROI, do not have the tools to prioritize hiring appropriately. How much should you pay for a data scientist is clearer if the business understands how much value that individual will return to the business.
The hype cycle bears a lot of blame for this problem. Companies are afraid of missing out on the benefits of data science and machine learning. Investors are starting to ask tough questions about how these emerging technologies will play into the larger strategy. Very few are talking about concrete solutions. Hype gets all the likes.
Better looks like these emerging technologies being integrated into business strategy as solutions to business problems. Better is oversight through executive sponsorship and a clear schedule of deliverables. Better leads to more concise requirements which goes a long way towards attracting top talent.
There are plenty of data scientists to go around. Clarity around how data science and machine learning solve business problems is not plentiful. That is the real skills gap.