Version Prompts, Models and Workflow Logic as One AI Release

An AI release is the complete behaviour-affecting configuration, not merely a prompt string or model endpoint. Rolling back the model while leaving a changed tool schema, permission rule, retrieval setting or retry path live does not restore the behaviour that was tested. Give the whole runtime bundle one identity, connect it to the evidence used to approve it, promote that same candidate through measured stages and keep a compatible known-good bundle ready. Otherwise, an incident team may know which model family served a request yet still be unable to reconstruct what the service was allowed to do.

What should count as one AI release?

One AI release should include every dependency that can materially alter served behaviour, authority, risk, cost, latency or observability. The inventory normally covers the prompt; resolved model identifier and inference parameters; tool contracts and permissions; policies or guardrails; retrieval and context settings; workflow code; input and output schemas; runtime dependencies; and environment bindings that affect request handling. NIST treats lifecycle activities and system components as interdependent, while Google Cloud describes production ML as far more than model code. The exact boundary nevertheless belongs to the service, not to a universal checklist.

Keep assurance assets distinct from the serving bundle but linked to its release record. Evaluation datasets, graders, rubrics and thresholds usually do not execute for a production request, yet changing them can change whether an otherwise identical candidate is authorised. Version relevant first-party and third-party dependencies only when they can materially affect the service or the release decision. A change to any effective prompt, provider snapshot, parameter, tool, permission, policy, retrieval setting, workflow, schema, runtime dependency or behaviour-affecting binding should therefore create a fresh candidate.

• Runtime identity: the resolved components and effective settings that determine how requests are handled.
• Assurance identity: the versioned scenarios, graders, rubrics and thresholds that determine how evidence is interpreted.
• Promotion history: exposure, timing, observations and decisions linked to the immutable candidate rather than written back into it.

How do you bind the behaviour stack into a release manifest?

Bind the stack by freezing an immutable manifest whose references resolve to the exact candidate intended for evaluation and promotion. Record the release ID, creation time, owner, target service, status, stop conditions, rollback owner and previous known-good release. Each component needs a stable version, commit, digest, content hash or equivalent reference together with its effective settings. A floating label such as production or latest is only a pointer: store the target it resolved to. MLflow's documentation illustrates why, because prompt versions can be immutable while aliases and attached model settings follow different mutability rules.

• Prompt version or content hash, including response-format settings.
• Pinned model snapshot and effective sampling, length and provider parameters.
• Tool schemas, permission policy, guardrail bundle and approved connections.
• Retrieval reference, context rules, workflow commit and input or output schemas.
• Runtime dependency lock, feature-flag identity, routing constraints and compatibility findings.
• Links to evaluation assets, results, approval, rollout plan and side-effect runbook.

Consider support-assistant-r18, an internal assistant that summarises tickets, drafts replies and creates a customer-relationship-management follow-up only after an agent approves it. Its manifest binds prompt p-42, model snapshot m-2026-07 and effective parameters, tool schema t-9, permission policy policy-12, workflow commit wf-a71, output schema reply-6 and the runtime dependency lock. Its release packet separately links evaluation suite eval-23 and grader versions. Record approved connections and routing without storing secrets. Name support-assistant-r17 as the rollback target only after checking its compatibility with the new optional due-date and escalation fields.

If something can change served behaviour—or the evidence authorising it—it needs a resolved identity in the release record.

What evidence should decide whether the candidate advances?

The candidate should advance only when evidence for that exact configuration supports an owned promote decision. Release notes should state the intended behavioural change, every changed dependency, affected scenarios and interfaces, permission or observability changes, evaluation evidence, known limitations, residual risk, rollout owner and compatible rollback target. For support-assistant-r18, that means explaining the revised escalation behaviour, multilingual and missing-context cases, changed CRM contract, continued human approval, rare mixed-language risk and the on-call owner responsible for restoration.

Run the checks that apply to the service: build and interface validation, workflow-specific task evaluations, safety and authority checks, tool-behaviour tests, reliability, latency and cost checks, plus comparisons with the current release across important segments and costly edge cases. Do not let one favourable average conceal a material contract, permission, safety or segment regression. Version the dataset, graders, rubric and thresholds so reviewers can tell whether the candidate changed, the judgement method changed, or both. Each gate must conclude with promote, hold or reject and identify its owner.

How should the same candidate move into production?

The same resolved candidate should move from controlled observation to progressively broader production exposure. Begin with shadow or non-actuating replay where feasible, disabling or sandboxing write tools and other consequential side effects rather than blindly repeating production actions. Then use an internal group, a sticky production cohort, expanded exposure and finally full traffic. At every stage, compare the agreed task, safety, tool, reliability, latency and cost signals with the current release. Offline evaluation and replay support the decision, but they do not replace measured online observation.

1. Replay representative recent cases with writes disabled and compare complete traces with the current release.
2. Expose the candidate to an internal operational group while retaining required approvals.
3. Route a sticky production cohort so requests do not switch candidates unpredictably.
4. Expand only after the owned evidence and observation requirements have been met.
5. Move to full traffic while preserving release-tagged monitoring and the known-good target.

Store cohort rules, exposure allocation, observation timing and the disposition as linked deployment events. A preapproved increase in exposure can advance the same candidate; changing its prompt, model setting, tool, permission, policy, retrieval setting, workflow, schema, dependency or behaviour-affecting binding creates another candidate. Choose traffic shares, evidence requirements and observation windows from service risk, traffic, detection latency and operational capacity, not copied conventions. Sticky canaries and shadow runs still may miss rare failures or conditions that their cohorts do not represent.

When should the release stop, and what must rollback restore?

Stop exposure immediately under predeclared hard conditions such as a safety or policy breach, unauthorised tool behaviour, a material contract failure or a severe reliability failure. Use service-specific limits for quality, latency, cost and other regressions because no universal threshold or observation period fits every workflow. An ambiguous result may justify pausing and investigating rather than automatically rolling back, but the disposition and owner must still be explicit. The decision should follow the evidence attached to the serving release, not an aggregate dashboard signal stripped of scenario context.

Rollback must restore traffic to the complete compatible known-good release, not revert one isolated component. Rehearse restoration before it is needed and check schemas, state changes, provider availability, tool contracts, routing and migrations before assuming the previous bundle still works. Google Cloud guidance recommends testing that a prior serving version can be restored and retaining the previous version in deployment metadata. For support-assistant-r18, restoration means returning to the checked r17 bundle rather than merely changing m-2026-07 back to an earlier model snapshot.

• Record the trigger, first observed time, affected release and production cohort.
• Identify affected requests, workflow paths and external action identifiers from release-tagged traces.
• Restore traffic to the compatible target and validate the service after restoration.
• Run separately authorised containment, reconciliation, correction, notification or compensating actions where required.

Configuration rollback governs future routing; it does not delete messages, reverse data writes, cancel approvals or undo completed actions in another system. If r18 created an unauthorised or malformed CRM task, disable the action path, locate the affected task identifiers through traces and follow the organisation's authorised CRM correction runbook. Reverting the configuration cannot remove an existing task by itself. Security, privacy, legal, records, risk or domain specialists should determine requirements when sensitive data, consequential permissions, regulated workflows or external-action remediation are involved.

What record makes an AI release reconstructable later?

A reconstructable release record preserves the effective configuration, the evidence considered and the decisions taken. Retain the immutable candidate manifest, resolved identifiers and parameters, environment bindings, compatibility findings, evaluation asset versions and results, approvals, deployment events, traffic allocations, traces, findings, rollback events and final disposition. Google Cloud recommends recording component and pipeline versions, execution details, parameters, artefacts, evaluation results and a pointer to the previous model. The release packet extends that lineage to prompts, tools, policies and workflow logic.

Attach the release ID to each trace so generations, tool calls, handoffs, guardrails, timings and outcomes can be attributed to the configuration that served the request. Record application trace identifiers and provider request IDs where available to support troubleshooting across system boundaries. Trace systems can capture rich workflow detail, but completeness does not require retaining every prompt, customer payload, tool input or output. Preserve governed identifiers and necessary evidence while applying the organisation's information-handling and retention policy to sensitive content.

Call the outcome configuration reproducibility and decision reproducibility, not guaranteed output reproduction. Resolved versions and archived settings can show what was intended to run and why it was promoted, but stochastic generation, hosted infrastructure and provider availability can prevent byte-identical replay. The smallest useful packet is therefore one that identifies the immutable runtime candidate, its versioned assurance evidence, its staged promotion decisions and its compatible rollback target. That record gives future operators a defensible reconstruction without pretending that an AI service is deterministic.