Building a Better Machine Learning Team

Machine learning has become a larger piece of business models. Revenue is increasingly dependent on more complex projects. The new Machine Learning Team is built to execute.

Vin Vashishta | Originally Published: August 15th, 2020

Book time with me for Career Coaching or sign up for my Business Strategy for Data Scientists class.

The team who gets the business through the prototype and proof of concept phases, is not the same as the team who will monetize machine learning. This key concept is a starting point for moving forward in the machine learning maturity model. So, what does “better” mean?

The business comes into machine learning expecting the technology to grow their bottom line. A better team creates the processes, relationships, and infrastructure to meet the need. Those are a combination of technical and soft skills. As the business’s machine learning projects mature business and domain knowledge become more important

Building a team will require hiring, promoting from within, and training.

Better Products Need Better Intrateam Coordination and Support

Relationships must come first. It may seem strange to start with the soft skill side of a very technical field. However, proof of concept projects do not build long term relationships between the machine learning team and business units. Those projects feel like one and done. The business unit is using machine learning. Their products are AI. Mission accomplished.

Six months, sometimes longer, after the first project, machine learning is a larger part of the business strategy. The machine learning team comes back to the business unit with a new project. There can be tension, either because the users do not understand how the project will help or because the first project failed in the dark.

There is a reluctance for business units to tell senior leadership about failures. Who wants to be the one to tell the CTO or Chief Data Scientist that their shiny new project flopped? Messengers get shot. They get called dinosaurs and told to adapt or die.

Building a better machine learning team starts with repairing neglected relationships. Rebuilding broken prototypes. Listening to business needs and how the first project fell short. The next machine learning project often involves cleaning up the first one.

Communications, collaboration, business acumen, relationship building, requirements gathering, many of these capabilities are missing in the first version of the machine learning team. The first data scientists are brought in for their strong technical skills. The soft skills were a nice to have and not a core piece of the team’s mission statement.

Analysts are part of the solution. They work closely with business units, so they have relationships and business acumen. They also have technical skills. They can be the technical connection between stakeholders and the machine learning team.

The machine learning team and other business units need structured, ongoing communication sessions. Machine Learning Scientists should talk about the technology with other business units. The goal is to explain what kinds of problems the technology solves. That understanding can lead to the business unit proposing projects so business needs are coming from teams rather than being dictated to teams.

Members of the business unit should talk about their roles in the business. They should share challenges and day to day realities. An open discussion about where technology helps and where it causes more problems than it solves is important. The goal here is a knowledge transfer about how the unit operates and how they provide value. The machine learning team can evaluate their technical solutions with an understanding of real-world operations. They start building for the user instead of for themselves.

Process and the Machine Learning Product Roadmap

This is the beginning of relationship building and repair. For a successful, long term relationship, the machine learning team needs to execute. They need to deliver products for customers and internal users. Machine learning products need to meet the same standards as any other product. The label does not exempt the team from on time, on budget, and as specified delivery.

The second phase machine learning projects are more complex and need more structured project management to achieve delivery goals. The first phase did not require much of a framework or cadence. The attitude was, “Let’s see if this works then go from there.”

It is hard to manage a project when no one outside the team understands any of the tasks. There is a layer of translation needed between the technical complexities and the managed process all other business units work with. New roles and experience are required.

Building a better machine learning team adds senior, distinguished, staff, principal; whatever the business’s specific label is, talent. These Machine Learning Scientists can take a project from problem statement to production. That does not mean they do all the work. Their efforts are focused on three steps that happen while building the machine learning product roadmap.

The Principal Machine Learning Scientist sits in on strategy planning meetings. They hear the business problems and goals for growth. What makes them different is their ability to hear a problem or goal and say, “Machine learning could be part of the solution.” The strategy team can prioritize problems/goals for technical evaluation based on that recommendation. Here’s where roadmap planning starts.

The Principal Machine Learning Scientist researches:

  • The Problem Space
  • The Solution Space
  • The Data

  • Each is a gate for evaluation. The problem space answers the questions, “Is this a machine learning problem? How much value can the business expect from a machine learning solution vs a traditional approach?”

    The solution space answers the questions, “What machine learning approach(es) can solve this problem? Does the team have the capabilities needed to implement that approach?”

    The data space answers the questions, “Does the business have enough/the right data to support the approach? What additional data needs to be gathered?”

    There is an unstructured component to machine learning projects. Research does not follow a set schedule or highly structured deliverables. Putting that piece into the planning phase prevents it from causing unknown timelines. This also puts control of green lighting each phase in the hands of senior leadership. Gate 1 defines the potential for revenue or cost savings. Gate 2 and 3 define project cost and give a high-level estimate of duration.

    Once all 3 gates have been met, the project gets put on the machine learning product roadmap. The Principal Machine Learning Scientist can provide the team with detailed guidance on how to complete the project. This framework makes the machine learning development lifecycle a managed process. The lifecycle phases and implementation follow a structured framework with customizations for each team and business.

    Infrastructure for the Path to Production

    The path to production for a proof of concept or early machine learning product is best effort. Back to the relationships part, when the machine learning team investigates why the product did not work, they often find it was not implemented as designed. Sometimes the model was not deployed at all and there is a rules engine where the model should be.

    The machine learning development lifecycle and path to production are connected. Infrastructure must support each phase. Building a better machine learning team adds people with deeper technical skills. A Machine Learning Architect and Machine Learning Engineer implement what is called Machine Learning Ops or MLOps. That term is a sales pitch for software but also the buzzword of the year.

    The Machine Learning Architect and Principal Machine Learning Scientist work closely together. They both have a complete view of what it takes to build, deploy, and maintain machine learning models. The Principal Machine Learning Scientist defines the infrastructure requirements. The Machine Learning Architect creates the architecture to support those requirements.

    Machine learning products do not exist in a vacuum. There are multiple points of integration. The machine learning architecture includes production and user environments. Integration extends to usability. Requirements from outside of the machine learning team must be supported.

    Architecture can sprawl to the point of becoming heavy, expensive, and disruptive to other groups. Simplification is the skill of a master architect.

    The Machine Learning Architect’s and Engineer’s first job is clean up. Machine learning teams often use, how do I say this nicely, worst practices. The prototype and proof of concept projects are completed using each data scientist’s tool kit of choice. Disparate tools are a barrier to collaboration. They become time sinks and money pits.

    One tool for each purpose. There are core tools that cover a large part of the machine learning architecture. Core tools are supported by an ecosystem of purpose-built tools. The Machine Learning Architect customizes their technology stack without having to build everything from the ground up.

    Custom model development leads to environment customizations. Connecting the machine learning tools environment with development and user environments requires further customization. Open source tools allow the machine learning team to add features supporting their specific development needs. This drives a few companies to contribute to existing open source projects or launch their own.

    The Machine Learning Engineer is responsible for implementation. Implementation starts with data gathering so their role supports the Principal Machine Learning Scientist. On the other side of the machine learning development lifecycle, they work with Machine Learning Scientists. They build optimization libraries for the team to use. These reduce model development, training, and validation times.

    The Machine Learning Engineer takes a completed model and integrates it with the development or user environment. After deployment, they work with Machine Learning Scientists to maintain and improve models.

    The scope of capabilities required to build the infrastructure and move machine learning products through the development lifecycle is broad. A single Machine Learning Engineer is rarely tasked with every step. As architectures become more complex, a diverse set of Machine Learning Engineers are needed.


    As hard as building the team sounds, the hardest part is building case for the expense and effort.

    This is the new machine learning team. It is integrated with the rest of the business. It has managed processes. Releases have cadence and meet the business need. At the end of this road, machine learning is a core competency to support the business model.

    There must be obvious, significant value for a business. Obvious is the most important part. The cost and effort can only be supported by belief in the business need and technology. What does it take for a retail, manufacturing, automotive, or medical company to decide to adopt a machine learning first business model? What would it take for, say Walmart, to shift their business model to become a machine learning focused technology company? I would say competing with Amazon would drive that shift.

    Not every business has an obvious threat to their continued operations from a machine learning first competitor. However, every business has a threat to their continued operations from a machine learning first competitor. I justify the expense to clients using the basic strategy tool, SWOT (Strengths, Weaknesses, Opportunities, Threats).

    Having a machine learning team does not mean the business can list machine learning as a strength. Building a better machine learning team moves the technology into the strength box. Having a machine learning team does not mean the business is able to execute and take advantages of opportunities (monetization). The complete machine learning team creates a competency around the technology. Infrastructure, managed process, cadence, a path to production; monetization is an unrealized goal without them.