AI governance by design means the controls an operation needs are built into the AI system from the first line of code: who may do what, which automated actions wait for a person, what is kept as a record, and how the system is watched once it runs. It is engineering work, done before go-live. At OrNsoft we call it AI trusted by design.
AI governance by design is an engineering approach that builds permissions, approval gates, traceability and production monitoring into an AI system before it goes live, so the operation can depend on it.
This pillar explains what that looks like in practice, layer by layer, with the flow of one sensitive automated action and the steps to put it in place. It is written for the people who answer for the result: the CIO, the CTO, the Chief AI Officer and the COO.
Key takeaways
- Governance that lives only in a policy document stops at the first automated action. Governance by design runs inside the system.
- Six principles cover it: Governed · Auditable · Observable · Secure · Model-neutral · Built to run.
- The control that matters most in operations is the approval gate, and your team decides which actions pass through one.
- Governance sits at five layers: people, workflows, models, data, operations.
- Public frameworks tell you what to manage. The engineering decides whether it holds on a live system.
What is AI governance by design?
It is the practice of turning governance rules into parts of the system. A rule such as “a person approves any refund above a threshold” becomes a gate in the workflow, a permission on a role and a record kept with the action. The rule then applies every time, at the speed the system runs.
Public frameworks describe what an organization should manage. The NIST AI Risk Management Framework, a voluntary framework, organizes the work into four functions: Govern, Map, Measure and Manage. ISO/IEC 42001 specifies requirements for an AI management system inside an organization. The OECD AI Principles name accountability as one of their values-based principles.
Each one asks for roles, oversight and monitoring. Governance by design is where those requests become code. This article covers the engineering. It is general information, and legal questions about a specific system belong with counsel.
Why does AI governance have to be built into the operation?
Because an AI system in operations acts. It drafts quotes, updates records, routes requests and triggers steps in other systems. Once it writes into the systems the business runs on, an error carries a real cost, and a rule written only in a policy document has no way to stop it. AI governance in operations has to act at the same speed as the system.
Operational AI governance answers five questions on a live system. The table shows the difference between answering them on paper and building the answer in.
| The question | Governance on paper | Governance by design |
|---|---|---|
| Who may do what? | A policy lists roles | Permissions per role, enforced by the system |
| Which actions wait for a person? | A guideline says “high-risk actions” | A gate per action, defined in an approval matrix |
| What happened, and why? | Reconstructed after the fact | Inputs, decision and approver kept with the action |
| Is it still working? | Checked at the pilot review | Monitored and evaluated in production against agreed criteria |
| Can the model change? | A new project | A swap behind the same controls |
The right column is work that has to be designed, built and owned. It is also one of the decisions that most often stalls a pilot: why AI pilots stall covers the missing approval path as one of five open decisions.
What are the six principles of AI trusted by design?
Every system we deploy for a client follows the same six principles, regardless of the model behind it.
| Principle | What we build | The question it answers |
|---|---|---|
| Governed | Permissions per role, approval gates for sensitive automated actions | Who may act, and what waits for a person? |
| Auditable | Automated actions and decisions traceable to their inputs | What happened, on what basis? |
| Observable | Monitoring and evaluation in production, beyond the pilot | Is it still doing the job? |
| Secure | Encryption in transit, tenant isolation, role-based access. Your infrastructure or ours | Does it pass a security review? |
| Model-neutral | Claude, GPT, Gemini, Mistral or open-source, switchable | Can we change the model without starting over? |
| Built to run | Engineered for sustained production use. A named senior lead accountable | Who answers for it once it is live? |
Model-neutral, which some call model-agnostic AI, matters for governance in a specific way. The controls belong to the system, so permissions, gates and records stay in place when the model changes.
How does an approval gate work for a sensitive automated action?
An approval gate is a point in the workflow where a sensitive automated action stops and waits for a person. AI approval gates are the control that lets an operation hand real work to an AI system and still decide what matters. Your team defines what the system may do autonomously and which actions require approval.
Take an illustrative example: an AI agent that drafts a revised quote for a client.
- The system proposes. It drafts the action and the data it is based on.
- The permission check runs. The system confirms that the role behind the action is allowed to take it.
- Your rules classify it. A quote above a threshold your owners set, or one that changes payment terms, counts as sensitive.
- A person decides. The reviewer sees the proposal and its inputs, then approves, edits or rejects it.
- The decision is kept with the action. Inputs, decision and approver stay together, so the question “why was this sent?” has an answer.
- Production monitoring continues. Gate outcomes are reviewed against agreed criteria, and the approval matrix is adjusted as the owners learn.
For high-risk AI systems, Article 14 of the EU AI Act requires a design that allows effective oversight by natural persons. An approval gate is one engineering measure that can support that kind of requirement; on its own, it does not make a system compliant. Whether a given system falls under it is a legal question.
Where does governance sit in each layer of the system?
At every layer. A gate in the workflow is only as strong as the roles above it and the data controls below it, so governance has to be designed through the whole stack.
| Layer | Control built in | Principle |
|---|---|---|
| People and roles | Permissions per role, an approval matrix signed by the owners | Governed |
| Workflows and actions | A gate on each sensitive action, with its record | Governed, Auditable |
| AI models | A replaceable model, with the controls kept in place | Model-neutral |
| Data and integrations | Encryption in transit, tenant isolation, role-based access | Secure |
| Operations | Monitoring against agreed criteria, a named senior lead | Observable, Built to run |
The hosting options and what we send on request are on our trust and security page.
How do you put an AI governance framework into a running operation?
In seven steps, before go-live. This is the order we follow in forward-deployed engineering engagements.
- Define production in writing. One page that says what production means, signed by the accountable business and technical owners. The template is in how to define production for an AI system.
- List the actions. The actions the system can take in other systems, each with its consequence if it goes wrong.
- Write the approval matrix. Which actions run on their own, which wait for a person, and who that person is.
- Set roles and permissions. Aligned with the matrix, so each role can do only what it allows.
- Build the gates and the records. In the workflow itself, one gate and one record per gated action.
- Monitor in production. Evaluate against the success criteria the owners signed, and review gate outcomes on a schedule.
- Name the owners. A named senior lead accountable until the system runs, a runbook and a documented handover.
The week-by-week sequence is in how a forward-deployed engineering engagement works. When an operation needs the system around the AI, the portals, workflows and shared data, a Platform build applies the same principles to the whole platform.
How does OrNsoft apply AI trusted by design?
OrNsoft is an advanced AI engineering firm, founded in Florida in 2009, with applied AI in its software since 2015. We build AI products and bring AI systems from concept or pilot to production for operations where accuracy, control and accountability matter.
We work with your existing codebase, build from scratch, or build on AdminCore, our proprietary technology; on our cloud, yours or on-premise; with the AI models you choose.
What should you do next?
Take one AI system you run, or plan to run, and answer four questions:
- Which actions can it take in other systems?
- Which of them wait for a person today?
- Who decided that, and where is it written?
- Who answers for the system next quarter?
Every “we do not know” is a governance decision to make before go-live. Book a call to go through it with us. One call is enough to know whether we can help.
Talk to Johnny Saffar, founder and CEO.
Questions
- Does AI governance by design slow an AI system down?
- Only where you choose. Actions outside the approval matrix run at full speed within agreed limits, and only the gated ones wait for a person. As the owners review gate outcomes, they can move an action out of a gate, or into one, without changing the rest of the system.
- What is an approval gate in an AI system?
- A stop in the workflow for one class of action. Two details decide whether it works in practice: the reviewer needs the inputs next to the proposal, or approving becomes a reflex, and the matrix should name a backup approver, so a gated action never waits on one person's calendar.
- Which automated actions should require approval?
- Your team decides. A practical starting point is any action that is hard to reverse, moves money, changes a commitment to a customer, or crosses a threshold the owners set. Start strict: releasing an action from a gate once the owners trust it is easier than adding a gate after an incident.
- What is the difference between an AI governance framework and AI governance by design?
- Use the framework to decide what to manage, and the system design to prove it. A simple test: for each control the framework asks for, someone should be able to point to where it runs in the live system. A control nobody can point to is still on paper.
- What does model-neutral AI mean?
- The model can be replaced while the controls stay. To test it, ask what has to change in the system to swap the model. If the answer includes the permissions, the gates or the records, the controls are tied to the model. Re-run the evaluation set after any switch, because a new model can behave differently on the same inputs.
- Does the EU AI Act require human oversight?
- For high-risk AI systems, yes: Article 14 asks for a design that lets people oversee the system effectively. Outside that category, oversight is still a sound design choice for actions where an error is costly, and the same approval gates serve both cases. Whether a given system is high-risk is a question for counsel.
Bring us the problem.
Book a call