Building an Internal AI Governance Framework

One 2026 analysis put the gap bluntly: 83% of organizations use AI tools, and only about a quarter have strong governance in place. The gap exists because most companies adopted AI one tool at a time. A team signs up for an assistant, someone builds a custom bot, a vendor quietly adds an AI feature to software already under contract, and nobody owns the full picture. A cross-functional council fixes that by giving AI decisions a single home.

Below is a step-by-step way to build one, drawing on the NIST AI Risk Management Framework and on how several operators run this in practice.

Pick a reference framework first

You do not need to invent the structure. The NIST AI RMF organizes AI risk work into four functions: Govern, Map, Measure, and Manage. It is voluntary, and no federal law requires private companies to adopt it, but enterprise customers increasingly ask about it in vendor questionnaires. ISO/IEC 42001 is the certifiable alternative, a management-system standard built on the same plan-do-check-act cycle as ISO 27001, so a company already certified there can extend existing audit machinery. The two overlap heavily, and an inventory and policy set built for one carries over to the other. Choose whichever fits your customers and regulators, and treat it as scaffolding for your AI governance framework rather than a checklist to complete.

Step 1: Charter the council

Give the council a written charter before it reviews anything. The charter should name the members, state what the council can approve, reject, or escalate, and set a meeting cadence. A workable core is four functions with distinct jobs:

IT and security assess data flows, access, integration, and vendor security. Legal reviews contracts, intellectual property exposure, privacy, and regulatory duties. HR covers employee-facing tools, training, monitoring concerns, and change management. Operations owns the business case, process fit, and day-to-day impact.

Add a named executive sponsor who can break ties, and a single accountable owner for each approved tool. HR's seat deserves emphasis. Vinay Gidwaney and Mike Sullivan of OneDigital argue that AI adoption has to be managed like onboarding a new hire and not like rolling out a CRM, and their book is aimed at HR leaders for that reason. Their company runs an internal staffing function for AI, with recruiters and onboarders. A council run only by IT will miss the people side of the decision.

Step 2: Build the inventory, including what nobody told you about

NIST's guidance treats inventory as the unglamorous core of governance, and it should include shadow AI that nobody formally approved. Survey every department and cross-check expense reports and single-sign-on logs against the answers. Then ask vendors which of your existing products now include AI features. Repeat the audit about every six months.

Do not forget what your own people build. Michael Domanic, Head of AI at UserTesting, describes a rollout where anyone could create a custom GPT with minimal guardrails, and where more than 600 were built across a roughly 800-person company within a single month. That approach drove adoption, and it also shows how fast an unmanaged inventory grows. Your register should include internally built assistants alongside purchased software.

Step 3: Tier each tool by risk

Sort every item by what it touches and what it can do. Useful inputs are the sensitivity of the data involved, whether outputs reach customers or regulators, and how much autonomy the system has. OneDigital's internal taxonomy offers a handy autonomy axis: coworkers, which help people think and collaborate, builders, which produce an asset such as a document or software, and agents, which carry out tasks on their own. They deliberately start teams with coworkers because agents need reliable data infrastructure underneath them. A council can borrow the same logic and apply heavier review as a tool moves from suggesting to acting.

Map your tiers to outside rules where they apply. The EU AI Act sorts systems into prohibited, high-risk, limited-risk, and minimal-risk, and a good inventory doubles as the starting point for that classification.

Step 4: Create an intake and audit checklist

Every new tool should pass through the same questions, scaled to its tier. A solid checklist covers:

Data handling. What data goes in, where it is stored, whether the vendor trains on it, and how deletion works. Residency and sovereignty. Where processing happens. PWL Capital, a Canadian wealth management firm, built its client-facing AI summary tool on Microsoft Azure Foundry specifically because of data-residency requirements for sensitive financial data, a choice driven by the data and not by model preference. Purpose limits. What the tool may access and for what stated purpose. Cillian, founder of the data governance company Ethica, builds on purpose-based access control, which decides in real time what data an agent can reach for a given purpose, and the same question works in a manual review. Context and fit. Whether buying makes sense. PWL built its meeting-notes assistant in-house because a unified internal data lake lets it draw on a client's whole financial picture, while a purchased tool would only see one transcript. Vendor dependency. The OneDigital team warns against building critical processes on one rented model, since a vendor can throttle or revoke access, and favors portable internal logic. Require a fallback for any tool a process cannot survive without. Exit. How you get your data out and what happens if the vendor is acquired or shuts down.

Step 5: Approve in stages, and turn approvals into settings

Avoid a single yes or no. OneDigital's pipeline for AI coworkers is a good model for staged approval. Each one starts from a written job description, then passes through three stages: intern, where it is tested against pass or fail criteria, apprentice, where it receives best practices, guardrails, and a human supervisor, and full-time, where it enters production. Underperformers go on a formal improvement plan and back to the workshop. Translate that into sandbox, supervised pilot, and production, with named exit criteria at each gate and the ability to revoke access.

Then enforce the decision in the product wherever possible. Brian Alvey, CTO of WordPress VIP, describes building a governance plugin that narrows a content editor's AI-expanded creative freedom to a customer's approved brand rules, for example locking one editor to two fonts and three colors while giving a trusted senior designer a wider set. Role-based limits like these work better than a policy PDF, because the rule travels with the tool. Alvey also calls the bottleneck for AI delegation the work of writing down a business's implicit rules in enough detail that an intern or an agent could follow them, a useful test for your own approval criteria.

Step 6: Track and review on a schedule

Keep a single register listing each approved tool, its risk tier, approval date, and accountable owner. Review high-tier tools at least quarterly and revisit anything when a vendor changes its terms or capabilities.

Measure what matters. OneDigital deliberately avoids measuring AI success by token usage, which rewards waste, or by hours saved alone, which can reward corner-cutting, and anchors its messaging on time returned to employees and better insights. Domanic's team reviews its ten highest-value internal AI implementations every quarter to build informal return-on-investment cases and decide what to scale. Pair that value review with a risk review of the same tools.

Step 7: Plan for incidents and retirement

Define what counts as an AI incident, who gets notified, and how fast. Fold it into your existing security response rather than creating a parallel process. Also write down how a tool is retired, including data deletion and access removal, since forgotten tools are a common source of risk.

Keep the front door easy to use

A governance process that feels like a wall will push people toward unapproved tools. Offer a fast lane for low-risk requests, publish expected turnaround times, and tell people what was decided and why. The goal of an AI governance framework is a safe route that is also the easiest route.

Stay current as the rules change

Standards, regulations, and vendor terms keep shifting. Subscribe to The AI Report for ongoing coverage of how organizations are governing AI, so your AI governance framework keeps pace with the tools it oversees.