Artificial Intelligence, Machine Learning, and Agile Practices

According to a study by Deloitte on the State of Artificial Intelligence (AI), 57 percent of respondents said that AI will transform their organization in the next three years. AI, a broad term that encompasses everything from rule-based software to self-driving cars, can be defined as any system that mimics human thinking by understanding its environment and taking actions towards achieving a goal. 

The part of AI that has been extremely successful in the last decade is Machine Learning. This is a set of algorithms and statistical methods that find patterns in the data and can predict or improve a desired outcome. The software, called a model, learns from past data without being explicitly programmed by a person—hence, the idea that the machine learns by itself.

With the quick adoption of Machine Learning in the enterprise, and the hiring of thousands of technical experts such as Data Scientists and Machine Learning Engineers, a gap became apparent: the need for leaders who have a basic understanding of Machine Learning and Artificial Intelligence was growing at a fast pace. 

The Machine Learning cycle is closely aligned with the Agile methodology. Agile was designed for situations that are very complex and non-static, and which we do not fully understand from the start. It provides a roadmap and a direction towards finding the answers, but it allows and encourages change. It provides structure and a framework that gives the team control over the project, even if there are unknowns in the planning.

For these reasons, I recommend using Agile to manage the execution of Machine Learning projects, following these phases:

1. Project Kickoff

When starting the project execution, a good practice is to create a project charter. Not only will it specify all the roles and responsibilities of the stakeholders, but also the proposed scope of the project and the criteria by which we will measure success. The project should have a cost analysis completed and the tools in place to measure its impact once delivered.

Another useful deliverable is a roadmap or rough project plan, which will specify the milestones and deadlines within the larger enterprise.

2. Data Exploration Phase

Since data is foundational to doing Machine Learning, any project starts with a phase called Exploratory Data Analysis (EDA), when the data scientists analyze the available data and prepare it for use in the models. One of the biggest challenges is that we simply don’t know what we will find in the data before the EDA, so any development tasks and even the scope of the project might not be fully defined until after this stage. 

A best practice is to plan a few research spikes in your project roadmap to get the team access to the data and time to analyze it. It is also recommended to have a Subject Matter Expert available to answer questions about data attributes and their business importance if needed.

3. Modeling Phase

It is normal practice for the Machine Learning team to try two or three different algorithms when creating the models. This could take an unknown amount of time. We suggest allowing for at least three iterations of modeling stories. 

Use INVEST, SMART, or other Agile techniques to create good stories to correctly execute and measure the work. Capturing Agile metrics as soon as the project starts helps measure the time to market, velocity, story lead times, and project predictability. 

4. Measure & Adapt

There are three types of metrics that we will use in a Machine Learning project: analytical, tactical, and strategic. Analytical metrics are those that the data scientists will use to make sure the model performs from a technical point of view, such as accuracy or model lift. Tactical metrics will measure the work done, team members engaged, deadlines met, and other standard Agile metrics and reports. These measurements will combine into a picture that will reflect how we have tracked against the strategic metrics we have started from and the business outcomes we have set to accomplish.

5. Managing Risk

Machine Learning models can be biased, or unfair. This can happen because of biased data, a biased way to train the model, or because the algorithms used are not easily explainable. As project managers and business leaders, we need to understand the risks associated with the work we are doing. If we are creating a model that will affect a person’s finances (for example, a credit limit decision), then we have to understand and manage the risk associated with it just like we do for any other similar project.

These are just a few of the challenges when managing Machine Learning projects. The “AI Fundamentals for Managers” course within our Master’s in Project Management at Georgetown provides our students with a deeper understanding and a broader approach to creating and executing a successful Artificial Intelligence and Machine Learning strategy in any organization.