“No man is an island entire of itself; every man is a piece of the continent, a part of the main.”
Data based innovation is delivered by agile teams, not a lone wolf Data Scientist.
I have many conversations with people lamenting the failure of a data science project. Here’s one from last week: “We have heaps of data, so we decided to hire a data scientist to get some kind of value from it,” followed by “we didn’t really get a return, the data scientist is gone now, and I struggle to get analytics projects going, as everyone’s cynical of their value.”
This is how it tends to happen :
- A decision was made to make better use of data
- It was assumed a data scientist would be required
- A job was posted on Seek and after a series of interviews a data scientist was selected with some R or Python skills
- They were pointed at the haphazard collection of data
- They started applying models and algorithms to respond to some vague problem you proposed
- They struggled to deliver anything useable
- At this point they were given new tasks that should be directed to the Business Intelligence team
- This work annoyed them and they went back on Seek looking for companies doing “real data science”
- They either found another job, or are still there as a grumpy BI analyst earning data scientist rates
- The ROI was never realised
I have heard variations of this scenario many times, and I believe it’s caused by a misunderstanding of the role of a data scientist and how data science is done. Success with analytics requires not just data scientists but an agile team that includes data wranglers, data analysts, visualisation experts, and subject matter experts. That’s a lot of new hires just to ensure success.
The reality is, you’re probably not ready for a data scientist as their skills only make up a third of the project’s effort. Most NZ businesses don’t generate enough pure modelling work for a data scientist, and if you do, you probably won’t have enough data ready for them full time. From what I see, most in-house data scientists are “at best” 50% productive and the rest of the time work on their own pet project or low level BI (at data scientist pay rates).
My advice… outsource data science initiatives early, as you can turn the resources “off and on” when needed. Eventually, when you know the ROI and have the required buy in, you can hire these people with a little more confidence.
And yes, the photo is one of my Data Scientists.