An AI go-live checklist is a list of questions that each need a written answer before a system launches. This one has 25, in five groups of five: what production means, the data and systems, how you measure, security and approvals, and who runs it afterward. If you cannot answer a question, you have found work that is still open, and you have found it before go-live, not after.
An AI go-live checklist is a set of questions about one AI system, each answered in writing by a named owner, that together show whether the operation can depend on it.
AI is easy to demo. Production is the hard part. This article is the question version of that idea, written for the people who answer for the result: the COO, the head of operations, the business owner. The one-page template that records the answers, and the readiness checklist that goes with it, is in how to define production for an AI system, and the path with its gates is in from AI pilot to production.
Key takeaways
- 25 questions, five groups of five. Each group covers one decision that pilots usually stall on.
- Every question needs a written answer and a named owner. “We think so” is not an answer.
- A “no” is information. Fix it, or sign a decision to accept the gap and say why.
- Answer together. The business owner and the technical owner read the same sheet.
What belongs on an AI go-live checklist?
Five groups, in the order the decisions get made. They follow the decisions that stall pilots, which we describe in why AI pilots stall: integration, security review, measurement, approvals and ownership, plus the definition that holds them together.
Site reliability engineering has a long habit of this. Google published its Launch Coordination Checklist, which groups the questions under headings such as architecture, monitoring, security, release process and rollout planning. An AI system needs the same discipline, with questions about measurement and approvals added.
Group 1: do you know what production means for this system? (questions 1 to 5)
Everything else is measured against this group. If it is weak, the other twenty answers have nothing to stand on.
- What business problem does the system solve, and which decision or process does it change?
- Who is the business owner, the person who decides what good enough means?
- Who is the technical owner, the person who decides how the system fits your architecture and security rules?
- What is in scope, and what is explicitly out of scope?
- Is there a signed production definition with a go-live date, signed by both owners?
A good answer to question 5 is a single page. The sections, and a template, are in the production definition guide.
Group 2: is it connected to the real data and systems? (questions 6 to 10)
A pilot often reads a copy of the data. Production reads the live systems, under the identity and permissions they require.
- Does the system read live data from the systems of record, or a copy someone exported?
- Which identity and permissions does it use in each system, and who granted them?
- Does it run in the environment where it will stay (your repository and your cloud, or on-premise), not in a sandbox that has to be ported later?
- What does the system do when a source is slow, down or returns something unexpected?
- Who owns each data source, and how will you hear when it changes?
The usual blocker is access, not code: a service account, a firewall rule, an owner for each source. Write the list early.
Group 3: can you show that it is good enough? (questions 11 to 15)
Measurement turns an argument into a number.
- What does good enough mean, in measurable terms, and who agreed it?
- Is there an evaluation set of real cases, with the expected result for each?
- Was the evaluation set frozen when the criteria were signed, so every version is compared on the same examples?
- What does the system do with cases it was not built for, and how do you detect them?
- How will quality be measured after go-live, and who reads the measure?
Engineering teams that run services in production work this way. The Google SRE book defines a service level objective as a target value or range of values for a service level, measured by an indicator. For machine learning, Google researchers published the ML Test Score, a rubric for production readiness covering testing and monitoring. The evaluation set gives an AI system its indicators.
Group 4: is it secure, and does a person decide where a mistake is costly? (questions 16 to 20)
Start the security review early: a review that starts at the end is the step that moves the go-live date.
- Is there a written security file: where the data flows, where the system is hosted, who can access what?
- Has a security reviewer outside the build team read it and signed?
- Has the system been tested against hostile input, including prompt injection if it uses a large language model?
- Which automated actions are sensitive (a payment, a message to a customer, a change to a record of reference), and is there an approval gate in front of each?
- For each gate: who approves, and what happens if nobody does?
The OWASP Top 10 for LLM Applications is a practical list for question 18, from prompt injection to excessive agency. The NIST AI Risk Management Framework organizes the wider risk work under four functions: Govern, Map, Measure and Manage. The result of question 19 is an approval matrix: which action, who approves, within what time.
Group 5: can the operation run it after go-live? (questions 21 to 25)
A system with no owner after launch drifts back into a pilot: it works, nobody trusts it, and nobody fixes it.
- Is the release staged, starting with part of the traffic, a team or a region, with criteria for widening it?
- Is a rollback written down, and has it been tried?
- Who is the named owner after go-live, and have they accepted the role?
- Is there a runbook: how to watch the measures, handle incidents, and approve changes to prompts, models and rules?
- What triggers a review of the system, and who decides?
How do you score an AI go-live checklist?
Give each question one of three answers, and attach the evidence to it.
| Answer | What it means | What you attach |
|---|---|---|
| Yes | The question is closed. | A document, a test result, a signature |
| Partly | Some of it is done. | What is missing, the owner, the date |
| No | Nothing is in place yet. | The owner who will close it |
The system is ready to go live when every question is a Yes, or when a “Partly” or a “No” has a written decision, signed by both owners, to accept the gap and why. There is no passing percentage: one open question about security or ownership can matter more than many closed ones.
Who should answer the questions?
Together, in one sitting if possible. The business owner answers groups 1, 3 and 4 (question 19 especially). The technical owner answers groups 2 and 5. A security reviewer outside the build team signs group 4. The engineers write the evidence; the owners sign it.
A sheet filled in only by the team that built the pilot can say yes to things nobody has asked it to prove.
What should you do with the answers you do not have?
Three choices, and only three. Close the gap and answer again. Accept the gap in writing, with both owners’ signatures and the reason. Or hold the go-live date until it is closed. A “no” found before go-live costs a conversation. Found in production, it costs an incident.
Where does forward-deployed engineering fit?
When the answers are missing and nobody owns finding them, a forward-deployed team works directly with your business and technical teams, learns how your operation actually runs, and takes ownership until the system runs. Group 1 is where it starts. How the engagement runs, gate by gate, is in how a forward-deployed engineering engagement works. The offer is on our forward-deployed engineering page.
What should you do next?
If you have a system close to go-live, answer the 25 questions with your two owners and see which ones stay open. If you would like a second reader, book a call. One call is enough to know whether we can help.
Questions
- What is an AI go-live checklist?
- An AI go-live checklist is a set of questions about one AI system, each answered in writing by a named owner, that together show whether the operation can depend on it. Ours has 25, in five groups: definition, data and systems, measurement, security and approvals, and operation.
- How do you know if an AI system is ready for production?
- When every question has an answer you can point to: a signed page, a test result, a review, a runbook. A question answered with "we think so" is still open.
- What should be on an AI go-live checklist?
- The production definition, live integrations, a frozen evaluation set with results, a security review, approval gates for sensitive automated actions, a staged release with a rollback, and a named owner with a runbook.
- Who should sign off an AI system before go-live?
- The business owner, who decides what good enough means, and the technical owner, who decides how the system fits the architecture and security rules. A security reviewer outside the build team signs the security file.
- What if we cannot answer some of the questions?
- Treat each one as open work. Fix it, or record a decision signed by both owners to accept the gap and say why. Finding the gap before go-live costs less than finding it in production.
- What is the difference between a pilot and a production system?
- A pilot shows the AI can do the task on the examples people tried. A production system is one the operation depends on every day, on live data, under the rules that apply, with a named owner.
Bring us the problem.
Book a call