2. Roles and Structure

An effective AI Centre of Excellence requires a carefully designed team with diverse skills spanning strategy, governance, engineering, ethics, and risk. This page outlines the core roles, their responsibilities, and how they work together.

Leadership and Governance

Head of AI CoE

Accountability: Overall strategy, operations, and value delivery of the CoE Reports to: CIO, CDO, CTO, or Chief AI Officer

Key responsibilities:

• Define and evolve the AI strategy and roadmap
• Chair governance forums (Executive Steering, Portfolio Council)
• Own the budget, headcount, and vendor relationships
• Communicate progress, risks, and wins to senior leadership

Skills: Strategic thinking, stakeholder management, AI/ML literacy, change leadership

Executive Sponsor(s)

Accountability: Remove blockers, secure funding, and drive organisational commitment Typically: CXO-level leaders from Technology, Data, Risk, or Business

Key responsibilities:

• Provide top-cover for strategic AI initiatives
• Resolve cross-functional conflicts and prioritisation disputes
• Champion the CoE internally and externally

PMO Analyst / CoE Operations Lead

Accountability: Orchestrate governance cadence, reporting, and portfolio tracking

Key responsibilities:

• Manage meeting schedules, agendas, and follow-ups
• Maintain the portfolio backlog and stage-gate tracker
• Produce dashboards on value, risk, and adoption metrics
• Coordinate audits and compliance evidence

Skills: Programme management, data analysis, process design

Product and Delivery

AI Product Owners

Accountability: Define business value, requirements, and success criteria for AI use cases Embedded in: Business domains (federated model)

Key responsibilities:

• Articulate the business problem and value hypothesis
• Prioritise features and backlogs for AI solutions
• Champion adoption and change management within the business

Skills: Product management, domain expertise, AI literacy

Domain Leads / AI Champions

Accountability: Own the AI roadmap and delivery for a specific business domain

Key responsibilities:

• Liaise between the domain and the CoE
• Ensure domain teams follow golden paths and standards
• Surface lessons learned and reusable patterns

Skills: Domain expertise, technical leadership, stakeholder management

Technical Enablement

AI/ML Architects

Accountability: Design reference architectures, patterns, and technical standards

Key responsibilities:

• Create golden paths for data ingestion, training, deployment, and monitoring
• Review solution designs for adherence to architectural principles
• Evaluate and approve new tools, platforms, and vendors
• Maintain the technical standards and non-negotiables

Skills: Solution architecture, MLOps, cloud platforms, security, scalability

MLOps Engineers / Platform Team

Accountability: Build and run the shared AI platform and CI/CD pipelines

Key responsibilities:

• Provision infrastructure, compute, and storage for AI workloads
• Implement CI/CD for model training, evaluation, and deployment
• Maintain model registries, experiment tracking, and feature stores
• Automate monitoring, alerting, and rollback mechanisms

Skills: DevOps, Kubernetes, Terraform, ML frameworks (TensorFlow, PyTorch), observability tools

Data Engineers

Accountability: Deliver high-quality data products that power AI solutions

Key responsibilities:

• Build data pipelines and feature engineering workflows
• Maintain data catalogues, lineage, and data contracts
• Ensure data quality, freshness, and compliance (GDPR, CCPA)
• Collaborate with AI teams on data requirements

Skills: Data engineering, ETL/ELT, SQL/Python, data governance

Applied Scientists / ML Engineers

Accountability: Develop, train, and optimise AI models

Key responsibilities:

• Experiment with algorithms and feature engineering
• Train and fine-tune models to meet performance targets
• Collaborate with domain experts to interpret results
• Document model behaviour and limitations

Skills: Machine learning, statistics, Python/R, experimentation, model tuning

Prompt Engineers / Evaluation Engineers

Accountability: Design, test, and refine prompts and evaluation frameworks for LLMs

Key responsibilities:

• Craft prompts that align with task requirements
• Build evaluation harnesses to measure accuracy, relevance, safety
• Red-team prompts to detect jailbreaks, hallucinations, and bias
• Maintain prompt libraries and best practices

Skills: NLP, LLM fine-tuning, evaluation metrics, adversarial testing

Risk, Ethics, and Compliance

Model Risk & Ethics Lead

Accountability: Ensure AI solutions meet ethical standards and risk thresholds

Key responsibilities:

• Define bias, fairness, and explainability standards
• Conduct model risk assessments and red-teaming exercises
• Review evaluation results and approve/decline model launches
• Maintain the responsible AI framework

Skills: Ethics, risk management, bias detection, regulatory knowledge

Security Architect

Accountability: Embed security best practices into AI solutions

Key responsibilities:

• Define security baselines (identity, secrets management, isolation)
• Conduct threat modelling for AI systems
• Review data access controls and encryption strategies
• Respond to security incidents involving AI

Skills: Cybersecurity, threat modelling, encryption, IAM

Privacy Counsel / Data Protection Officer

Accountability: Ensure AI solutions comply with privacy regulations (GDPR, CCPA)

Key responsibilities:

• Conduct Data Protection Impact Assessments (DPIAs)
• Review data handling, retention, and anonymisation practices
• Advise on lawful bases for processing personal data
• Respond to data subject requests (access, deletion)

Skills: Privacy law, GDPR/CCPA, data minimisation, legal risk

Compliance Lead

Accountability: Maintain audit trails and ensure regulatory readiness

Key responsibilities:

• Track compliance obligations (AI Act, sector-specific regs)
• Maintain evidence packs for audits
• Coordinate with internal audit and external regulators
• Document decisions, approvals, and justifications

Skills: Compliance management, audit, documentation, regulatory frameworks

Vendor Manager

Accountability: Manage AI vendor relationships, contracts, and risk

Key responsibilities:

• Evaluate vendors on capability, cost, and trustworthiness
• Negotiate contracts with fair terms and exit clauses
• Monitor vendor performance and SLAs
• Maintain vendor scorecards and risk registers

Skills: Vendor management, procurement, contract negotiation, risk assessment

Organisational Structure

The CoE can be structured as:

Option A: Central Team + Federated Delivery The CoE is a central function (reporting to CIO/CDO) with embedded AI champions in business domains.

Option B: Virtual CoE No dedicated CoE team; instead, representatives from Technology, Risk, Data, and Business meet regularly to govern AI.

Option C: Hybrid A small central core (Head, Architects, MLOps) with federated domain teams and part-time subject-matter experts.

Most common: Option A or C, depending on organisational size and AI maturity.

RACI Example: Model Deployment

R = Responsible, A = Accountable, C = Consulted, I = Informed

Staffing Guidance

Next Steps

With roles defined, the next step is establishing the governance cadence — the meetings and decision-making forums that keep the CoE running smoothly.