Setting the charter for an AI Center of Excellence

“We created an AI Center of Excellence, now what?”

Built it, and They Will Come

At AlignAI, we get the opportunity to work directly with the teams and leaders living at the center of their company’s AI initiatives. Lately, it seems like just about every enterprise is standing up their own internal ‘AI CoE’, or ‘AI Council’, or 'AI Governing Committee’, or 'AI Steering Committee’, etc.  

The names vary, but the mission is consistent - a cross-functional team centralizing the expertise, tools, and best practices necessary for successful AI adoption. It serves as the nerve center for AI initiatives, guiding the organization through the complexities of AI deployment while ensuring that projects align with business objectives. 

We can see why enterprises form these groups - as teams rush to adopt AI, many encounter challenges in scaling their initiatives, ensuring consistent quality, and aligning AI projects with strategic business goals. This is where AI Centers of Excellence (CoE) come into play.

‍

Who is Invited?

In considering the org-wide implications of AI - leaders from HR, L&D, Legal, Procurement, Finance, Cybersecurity, Data Governance, Architecture, Business Functions and the Project Management Office (PMO) have a seat at the table.  These leaders are usually led by an Executive Sponsor to provide strategic direction and ensure alignment with organizational goals. Followed by an AI CoE Leader who manages day-to-day operations, coordinates with stakeholders, and leads the AI team.  

Naturally, there are two CoE sub-groups that begin to emerge.  The ‘build/platform team’ focused on solution development and the ‘governing team’ focused on risk reduction. Both are critical to ensure AI is done right, and there is a natural tension between these two sub-groups as one side prioritizes throughout while the other prioritizes thoroughness. More on that later...

‍

Why a Charter Makes Sense

An AI CoE charter starts by defining the purpose and vision of the CoE. It answers key questions: 

• Why does the organization need an AI CoE? 
• What business objectives will it support?
• What are the goals?
• What does the operating model look like?

By clearly articulating the purpose, the charter sets the tone for the CoE's operations and ensures that all stakeholders align on the broader vision. This clarity of purpose is essential for maintaining focus and driving the organization toward its AI-related goals. Without a well-defined vision, AI initiatives risk becoming fragmented, with no clear direction or measurable outcomes.  

Early on, CoE members can get hung up on pointing out all the areas where AI may go wrong instead of seeking ways to resolve it.  It is important to bring up risks and concerns, but more important to bring solutions. The charter helps to guide members in this direction of progress over perfection.  

The charter should define the following for the CoE:

• Mission Statement - Defines the purpose and overarching goals of the AI CoE
• Objectives - Lists specific, measurable targets to achieve the mission
• Scope and Activities - Delineates the boundaries and focus areas of the CoE's work
• Meetings/Communications - Outlines frequency and format of team interactions and updates
• Governance Structure - Defines the organizational hierarchy and decision-making framework
• Roles and Responsibilities - Specifies duties and expectations for each team member
• Decisions Supported by theCoE - Clarifies the types of organizational decisions the CoE will inform
• Resource Allocation - Details how personnel, budget, and technology assets will be distributed
• Performance Metrics - Establishes key indicators to measure the CoE's success and impact
• Risk Management - Identifies potential challenges and mitigation strategies
• Review and Revision - Sets schedule and process for updating the charter as needed

There is a lot to define in a charter for a newly formed team, so high performing CoEs start by focusing their efforts on the relevant use cases being brought up by the business partners, helping to:

• Identify and prioritize AI use cases
• Conduct AI research and pilot projects
• Establish best practices and standards for AI development and deployment
• Vendor Evaluation & Onboarding Support which includes AI
• Provide training and support for AI initiatives
• Support compliance with ethical standards and regulatory requirements

‍

Not everything has to be brand new, many CoE’s start from a baseline for each of these functions and adjust as they go.  As the CoE works through the relevant use cases and ideas for AI at their company, patterns start to emerge and processes are naturally defined.  This fills in the gaps of the charter listed above as the team gets practice and learns what works. The biggest challenges that come up are the foundational data management elements that are not in place (ex. data classification). The goal is to demonstrate quick wins with results while managing as much risk as possible.  

‍

Eventually, the AI CoE expands into additional areas of focus with more resources and budget to support them, including:

• Developing and deploying AI solutions - by providing centralized resources to support AI development initiatives for business functions
• Monitoring and evaluating AI performance and impact - by designing an AI solution monitoring and evaluation platform

‍

Throughput & Thoroughness

The relationship between throughput and thoroughness in reviewing AI use cases is a classic trade-off that AI CoEs often need to balance, especially when trying to scale AI initiatives. 

A well-crafted charter is not just a formality; it is a foundational element that can make or break the success of an AI Center of Excellence. By defining purpose, scope, governance, and success metrics, the charter provides the framework needed to guide the CoE’s activities and ensure alignment with broader business objectives. Success metrics are especially important, below are the ones that successful CoE’s focus on:  

• Use Cases Identified - # of high-value use cases scoped by the team 
• Usage - # of employees/users leveraging AI solutions in their work
• Impact - Total financial impact tied to deployed AI solutions
• Release Mgmt - Time it takes to deploy an AI solution in to production
• Responsibility - # of AI solutions meeting internal quality standards set by the CoE

A charter helps to streamline processes and make efficient use of CoE resources, while also ensuring that AI use cases are reviewed with the necessary depth and rigor. For enterprises looking to harness the power of AI at scale, investing the time and effort to develop a comprehensive AI CoE charter is an essential first step.

For more on structuring AI committees for the enterprise, Rehgan Bleile has a great conversation with Noelle Russel on the topic here: https://youtu.be/w3maMQi6CQ8?si=gj0DLqSNPlz7TXdS

‍