The role of the data scientist is one of the most coveted jobs today. On LinkedIn, there are listings for over 52,000 companies looking for a person with these skills right now, with the majority (70%) listed as mid-senior level positions. Mashable called the role of the data scientist the hottest profession in 2015. And in a Harvard Business Review article, Babson College professor Tom Davenport called this position the sexiest job of the 21st century.
To find out more about this role and how it’s evolving with the onslaught of Internet of Things (IoT) data Professor Davenport, a fellow at MIT Center for Digital Business and a senior advisor to Deloitte Analytics, shared some insights into what makes the data scientist so critical today.
“It’s difficult to define a data scientist in day-to-day practice. Early on, there was a fairly clear distinction between the traditional quantitative analyst and a data scientist, but less so now,” says Davenport. “The traditional quantitative analysts had a primary focus of taking structured data and performing statistical analysis on that. It was a very creative and experimental kind of job in those early days.”
Today’s data scientists must not only have quantitative analysis skills, but they must also deal with difficult and unstructured data formats that play a key part in the predicative nature of data science. Davenport notes that early on, the process of converting unstructured data into structured data was difficult and time consuming, but technology is making this easier today. “There are new approaches to data analysis and data management that are more automated,” says Davenport. “Data scientists can now use things like machine learning that can generate models in a somewhat automated fashion.”
However, working with the volumes of data in the IoT space is still a challenge and requires strong data skills. For instance, there are typically many different types of IoT sensors within one single product, and they all gather and store data in different formats. “If you want to analyze all the data from a car, for example,” says Davenport, “you may be looking at more than 200 different sensors. The great majority of them – most likely 95% or more – have different data formats in which they store and format the data.”
This is where hardcore data science skills come in, says Davenport. “A data scientist needs to take all those diverse formats, pull them together, integrate them with other types of data, and analyze them in a way that makes the data useful.” From there, data scientists can provide insights that enable people to understand how a product performs under different conditions. They can see the likelihood and timing of needed maintenance, and identify design faults that may make a product break down frequently.
Another factor that comes into play when thinking of the role of a data scientist is the need for industry expertise in highly specialized segments. “A specific sensor application could be very detailed and idiosyncratic, which is where specific business insight is needed,” say Davenport. In cases like this, there may be a need for a data scientist to have other expertise specific to a certain industry or product. For instance, in the manufacturing of a unique part, it may be important for a data scientist to understand material science. That expertise may be critical in ensuring that the right sensors are made and the right data is collected for the anticipated analysis.
So how can data scientists best be utilized in an organization? Davenport suggests that they collectively work together with engineers to understand how the data generated from products will be used. When done properly, this actually enhances the job of analyzing the data and makes it easier. “I would also recommend giving data scientists full access to the engineering problems,” Davenport says, “and make them a member of your team. After all, engineers today no longer just develop products – they are in a much more expanded role. They are developing data- and analytics-based services that accompany that product and help the customer use it more effectively.”
Davenport also recommends that the data scientists have good relationships with the business people who are in the application domain. This helps them get full access to the business problem that’s involved. “Once they understand what the organization is trying to accomplish,” says Davenport, “it’s much easier to solve problems and present ideas.”
With the high demand and specific suite of skills needed, it’s no wonder that this is a function that is in high demand. “When you add highly specific knowledge together with the requisite ability to manage, integrate, and analyze data,” Davenport says, “it’s a tough combination. Add in the ability to be a team player and communicate what it’s all about to managers who need to make decisions, and it’s an even tougher combination.”