What Gets Measured Gets Done: How to Choose AI Projects that Deliver Value in 2024
After a 2023 full of AI pilot programs, a new executive order, and more than a little discussion around how organizations implement AI, it’s time to put up or shut up. The momentum behind AI is undeniable and organizations across public, private, and academic sectors are looking for ways to create new value capitalizing on its rapid rise. While 2023 was the era of small-scale pilot projects and questions surrounding further policy or regulatory guiderails, 2024 is the time for demonstration of value. But how you achieve that value is challenging, so how do you pick the right AI projects and determine that they are creating value?
In this blog post, I will share four criteria that you should consider when selecting and prioritizing AI projects. These criteria are based on a speech that I gave recently at JP Morgan on behalf of one of FF’s banking partners, where I discussed the best practices and toughest challenges of AI adoption in the business world.
1. Quantifiable Measure of Value
The first criterion is that your AI project should have a quantifiable measure of value that directly links to the financial bottom line of your company. This means you should be able to define and track the key performance indicators (KPIs) that reflect your AI project's impact on your revenue, profit, cost, or customer satisfaction. These should be defined before the start of your AI project (preferably before the budget is approved) and built into the measures of success. If possible, create two measures – one that is unique to the project and one that can be shared across multiple projects so that you can establish a baseline for measurement of value creation.
For example, if you are building an AI system to improve customer service, you should measure how it affects metrics such as customer retention, churn rate, net promoter score, or average resolution time. If you are developing an AI solution to optimize your supply chain, you should measure how it affects metrics such as inventory turnover, delivery time, or waste reduction.
By having a quantifiable measure of value, you can set clear and realistic goals for your AI project, monitor its progress and performance, and evaluate its return on investment (ROI). You can also communicate the value of your AI project to your stakeholders, such as your executives, investors, or customers, and justify your budget and resources.
2. Primary Business Sponsor
The second criterion is that your AI project should have a primary business sponsor, not an IT sponsor. This means that you should have a senior leader or a decision-maker from the business side of your organization who is responsible for and committed to your AI project.
The primary business sponsor should have a deep understanding of the business problem that your AI project is trying to solve, the data and domain knowledge that are required, and the expected outcomes and benefits. The primary business sponsor should also have the authority and influence to align the AI project with the strategic vision and goals of your organization, secure the necessary support and collaboration from other departments or teams, and overcome any potential barriers or risks.
Having a primary business sponsor is crucial for the success of your AI project, as it ensures that your AI project is aligned with the business’ needs and priorities, has a clear and compelling value proposition, and has the buy-in and trust from the key stakeholders. It also introduces accountability and creates a culture where team members outside of your IT or technology teams feel bought in to the process and the project’s ultimate success.
3. Delivery Window for Value
The third criterion is that your AI project should have a delivery window for value that is less than six months out. This means that you should be able to deliver a minimum viable product (MVP) or a proof of concept (POC) of your AI project within six months or less from the start date.
The delivery window for value is important for several reasons. First, it helps you to focus on the most critical and feasible features and functionalities of your AI project and avoid scope creep or over-engineering. Second, it enables you to test and validate your AI project with real users and data and collect feedback and insights that can help you to improve and iterate. Third, it allows you to demonstrate the value and potential of your AI project to your stakeholders and gain their confidence and support for further development and deployment.
By having a delivery window for value, you can adopt an agile and lean approach to your AI project and deliver value faster and more effectively.
4. Regulatory, Compliance, and Operational Envelopes
The fourth criterion is that your AI project should fit inside the regulatory, compliance, and operational envelopes of your organization. This means that you should ensure that your AI project complies with the relevant laws, regulations, standards, and policies that govern your industry and your organization, and that it operates within the technical and ethical boundaries that you have defined.
For example, if you are using personal or sensitive data for your AI project, you should comply with the necessary data protection and privacy regulations, such as the General Data Protection Regulation (GDPR) or the California Consumer Privacy Act (CCPA). If you are deploying an AI system that affects human lives or well-being, such as in healthcare or transportation, you should comply with the safety and quality standards, such as the Food and Drug Administration (FDA) or the Federal Aviation Administration (FAA). If you are creating an AI system that makes decisions or recommendations, such as in finance or education, you should ensure that it is fair, transparent, accountable, and explainable. In many cases those envelopes are either not defined yet or inconsistently applied – planning 3-5 years out should be included in this process.
By fitting your AI project inside the regulatory, compliance, and operational envelopes, you can avoid legal, ethical, or reputational risks, and ensure that your AI project is trustworthy and responsible.
Conclusion
Questions about the value proposition of any number of AI applications will continue to dominate AI discussions in the absence of measurable, quantifiable results from AI projects. AI’s power and utility are not in dispute but how an organization derives value from those attributes is. There is both a risk of using AI improperly and of NOT using AI at all when the situation calls for it. Organizations that value their reputation and their ability to stay ahead of competitors must consider both sides. Having enough data to do so accurately requires AI projects that are chosen and measured according to the criteria above. AI projects with measured results tied directly to business cases will provide value and the lessons learned to grow that value 10X in the next AI project.