Research shows that AI development and Agile methods don’t mix well

Agile Software Development has long been recognized as a very effective way to deliver the software that businesses need. This practice has worked well in many organizations for over two decades. Agile is also the foundation for melee, Development activitiesand other collaborative activities. However, agile activities may not achieve artificial intelligence Design and implementation (AI).

That insight comes from something recent. report from the RAND Corporation, a global policy research organization, based on interviews with 65 data scientists and engineers with at least five years of experience building AI and machine learning models in industry or academia. The study, originally conducted for the U.S. Department of Defense, was completed in April 2024. “Often, AI projects fail or never get off the ground,” the report’s co-authors, who were James RyseffSenior Technical Policy Analyst at RAND.

Also: Agile development can unlock the power of generative AI — here’s how

Interestingly, some AI experts consider formal agile software development practices to be Barriers to AI successThe researchers found that “Some interviewees (10 out of 50) expressed the belief that a rigid interpretation of agile software development is not appropriate for AI projects.”

“While the agile software movement never intended to develop rigid processes — one of its key tenets is that individuals and interactions are much more important than processes and tools — many organizations require their engineering teams to follow the same agile process.”

As a result, as one interviewee put it, “work items constantly have to be reopened in the next sprint or made unreasonably and pointlessly smaller to fit into a one- or two-week sprint.” AI projects in particular “require an early phase of data exploration and experimentation of unpredictable duration.”

Also: How Your Business Can Best Harness AI: Tell Your Board These 4 Things

RAND research suggests other factors may limit the success of AI projects. While IT failures have been well documented over the past few decades, Failure of AI have an alternative look. “AI appears to have different project characteristics, such as high labor and capital requirements and high algorithmic complexity, that make them unlike traditional information systems,” the study’s co-authors said.

“The emergent nature of AI may lead stakeholders to want to better understand the risks inherent in AI-related IT projects.”

The RAND team identified the main reasons why AI projects fail:

• “Industry stakeholders often misunderstand — or miscommunicate — the problem to be solved with AI. Often, organizations deploy trained AI models only to discover that the models optimize for the wrong metrics or are inappropriate for their overall workflow and context.”
• “Many AI projects fail because organizations lack the data needed to adequately train an effective AI model.”
• “This organization is more focused on using the latest and greatest technology than solving real problems for their intended users.”
• “Organizations may not have the infrastructure in place to manage data and deploy complete AI models, which increases the likelihood of project failure.”
• “This technology is applied to problems that AI cannot solve. AI is not a magic wand that can make every difficult problem a reality; in some cases, even the most advanced AI models cannot automate a difficult task.”

While formal agile practices may be too cumbersome for AI development, they are still important for IT and data professionals. Open communication with business users. Interviewees in the study recommended that “instead of adopting established software engineering processes — which are often just elaborate to-do lists — engineering teams should regularly communicate with business partners about the status of the project.”

Also: It’s time for businesses to get past the generative AI fever and find real value

“Stakeholders don’t like it when you say, ‘It’s taking longer than expected; I’ll get back to you in two weeks,’” the report suggests. “They’re curious. Open communication builds trust between business stakeholders and the engineering team, and increases the likelihood that the project will ultimately be successful.”
Therefore, AI developers must ensure that the technical team understands the project’s purpose and domain context: “Misunderstandings and miscommunication about project intent and goals are the most common reasons for AI project failure. Ensuring effective interaction between technology experts and business experts can make the difference between the success and failure of an AI project.”

The RAND team also recommends choosing “persistent problems.” AI projects require time and patience to complete: “Before starting any AI project, leaders should be prepared to commit each product team to solving a specific problem for at least a year. If an AI project isn’t worth that long of a commitment, it’s probably not worth committing to at all.”

Also: When Is the Right Time to Invest in AI? 4 Ways to Help You Decide

While it is important to focus on the business problem rather than the technology solution, organizations must invest in infrastructure to support AI efforts“Upfront investments in infrastructure to support data governance and model deployment can significantly reduce the time required to complete AI projects and can increase the volume of high-quality data available to train effective AI models,” the RAND report suggests.

Finally, as noted above, the report argues that AI is not a magic wand and has limitations: “When considering a potential AI project, leaders need to engage technical experts to assess the project’s feasibility.”