Version Prompts, Models and Workflow Logic as One AI Release

An AI release is the complete behaviour-affecting configuration, not a model endpoint or prompt string considered alone. Rolling back only the model can leave a changed prompt, tool schema, permission rule, retrieval setting or retry path serving requests. Without one identity for that full stack, operators cannot reliably show what they evaluated, determine which configuration handled a problematic request or restore the compatible setup that was known to work.

What should count as one AI release?

One AI release should contain every resolved dependency that can materially change served behaviour, authority, risk, cost, latency or observability. Production AI systems extend beyond model code, and lifecycle decisions interact with deployment conditions. The practical boundary is therefore the service behaviour that operators can observe and govern, not whichever object happens to live in a model or prompt registry.

• Prompt content and template version.
• Resolved model snapshot and effective inference parameters.
• Tool schemas, permissions, approval rules, policies and guardrails.
• Retrieval sources, context assembly and routing configuration.
• Workflow code, input and output schemas, and runtime dependencies.
• Feature flags, approved connections and other behaviour-affecting environment bindings.

Evaluation datasets, graders, rubrics and thresholds belong in the release record as versioned assurance assets. They usually do not execute in the serving path, but they can change whether a candidate is authorized to advance. Include first-party or third-party dependencies only when they can materially affect the service or its release decision. Any relevant component change starts a new candidate; component-level histories remain useful, but the release identity prevents emergency reconstruction from becoming detective work.

How do you bind the behaviour stack into a release manifest?

Bind the stack by freezing an immutable candidate manifest and putting changing rollout facts in linked deployment records. The manifest needs a release ID, creation time, owner, target service, status, stop conditions, rollback owner and previous known-good release. For every component, record a stable reference—such as a registry version, commit, artifact digest or content hash—plus the effective settings used by the candidate.

• Resolve floating aliases such as production or latest before recording the candidate.
• Record model parameters even when a prompt registry stores them, because some attached settings can remain mutable.
• Name approved connections, retrieval references, feature-flag identities and routing constraints without storing secrets.
• Document compatibility constraints, migrations and the evidence needed before restoration.
• Link evaluation-suite and grader versions without treating them as runtime components.

Consider an internal support assistant that summarizes tickets, drafts replies and creates a customer-relationship-management follow-up only after an agent approves it. Candidate support-assistant-r18 binds prompt p-42, model snapshot m-2026-07 and its parameters, tool schema t-9, permission policy policy-12, workflow commit wf-a71, output schema reply-6 and the runtime dependency lock. Its release packet separately links evaluation suite eval-23 and the grader versions.

The rollback target is support-assistant-r17 only after a compatibility check covers the candidate's optional due-date and escalation fields. Exposure allocation and deployment times should remain linked events, not mutations to r18. A preapproved allocation increase can advance that candidate; changing its prompt, authority, context, routing behaviour or another effective binding produces a different candidate with its own evidence.

If something can change served behaviour—or the evidence authorizing it—it needs a resolved identity in the release record.

What evidence should decide whether the candidate advances?

The candidate should advance only when evidence from the exact complete configuration supports an owned promote decision. Release notes should state the intended behaviour, every changed dependency, affected scenarios and interfaces, permission or observability changes, known limitations, residual risk, rollout owner and compatible rollback target. That decision record should make a hold or rejection as explicit as an approval.

• Run build, schema, dependency and serving-interface checks that apply to the workflow.
• Compare task quality with the current release using realistic cases, important segments and costly edge cases.
• Test safety boundaries, authority rules, tool arguments, approvals and prohibited actions.
• Check errors, latency, resource use, cost and trace readiness against service-owned limits.
• Version datasets, graders, rubrics and thresholds, then record who approved the disposition.

A single average must not conceal a material contract, safety, authority or segment regression. General model benchmarks cannot capture every business workflow, and automated graders need audit and domain oversight. Higher-risk releases can separate the change author from the promotion approver; smaller teams may combine roles, but they should still retain the evidence, decision owner and any exception.

How should the same candidate move into production?

Move the same resolved candidate through progressively more representative stages while recording each exposure decision. Offline evidence and production observation answer different questions: predeployment checks can expose known failures cheaply, while live use reveals interactions with actual traffic and operating conditions. Neither a shadow run nor a limited cohort proves complete safety or coverage.

1. Replay representative work in shadow or non-actuating mode, with write tools and consequential side effects disabled or sandboxed.
2. Expose the candidate to an internal group while preserving required approvals.
3. Assign a sticky production cohort and compare task, safety, tool, reliability, latency and cost signals with the current release.
4. Expand exposure only after the service's observation and evidence requirements are met.
5. Move to full traffic while retaining release-tagged monitoring and the compatible rollback target.

Keep cohort rules, traffic allocation, observation timing and disposition in deployment events linked to the candidate. Choose allocation, sample requirements and observation windows from service risk, traffic volume, detection latency and operational capacity; universal percentages would create false precision. Sticky assignment improves comparison, but canary traffic may still miss rare failures or conditions absent from the selected cohort.

When should the release stop, and what must rollback restore?

Stop exposure when a predeclared hard condition is met, and restore traffic to the complete compatible known-good release. Hard stops should cover safety or policy breaches, unauthorized tool behaviour, broken contracts and severe reliability failures. Other regressions need service-specific limits and observation rules. An ambiguous finding may justify a pause and investigation instead of automatic rollback, but the disposition and owner must still be recorded.

Rehearse restoration before it becomes urgent. Confirm that the prior bundle still works with current schemas, state, provider availability, tool contracts, routing and migrations; a label such as known-good does not guarantee present compatibility. Restoring only the model can preserve the prompt, policy or workflow change that caused the fault, so traffic should return to the tested bundle as a whole.

• Record the trigger, first observed time, affected release and cohort.
• Identify affected requests, workflow paths and external action IDs from release-tagged traces.
• Restore traffic and verify the known-good bundle against current conditions.
• Contain or reconcile completed actions through a separately authorized runbook.
• Record the final disposition, follow-up owner and resulting evaluation or monitoring changes.

Rollback controls future routing; it does not delete sent messages, reverse data writes, cancel approvals or remove tasks already created in another system. For the support assistant, operators could disable task creation, find affected task IDs in traces and follow the approved customer-relationship-management correction process. The required containment, notification, restoration or compensating action depends on organizational authority and the action involved.

What record makes an AI release reconstructable later?

A reconstructable release record links the immutable manifest to the evidence and decisions that moved it through production. Retain resolved component identifiers and parameters, environment bindings, compatibility findings, evaluation asset versions and results, approvals, deployment events, traffic allocation, significant traces, findings, rollback events and final disposition. Include provider request identifiers and application trace identifiers where available to help connect symptoms across system boundaries.

• Manifest identity, owners, timestamps and resolved components.
• Evaluation methods, asset versions, results, limitations and approvals.
• Cohort rules, deployment events, observations and exceptions.
• Release-tagged traces, request identifiers and relevant action identifiers.
• Rollback compatibility, restoration evidence and final disposition.

Attach the release ID to traces so generations, tool calls, handoffs, guardrails, timing and outcomes can be attributed to the configuration that served them. This does not require retaining every prompt, tool input, model output or customer payload. Sensitive payload capture and retention should follow organizational policy; identifiers, outcomes and governed samples may provide the necessary operational evidence with less exposure.

Call the result configuration reproducibility and decision reproducibility, not guaranteed output replay. Pinned snapshots, hashes and archived settings establish what operators intended to run and why they authorized it, but stochastic sampling and hosted-service changes can prevent byte-identical results. The useful standard is a defensible reconstruction of configuration, evidence, exposure and decision.

Start with the smallest packet that still identifies the candidate, its assurance evidence, promotion decisions and compatible rollback target. Bring in qualified security, privacy, legal, records, risk or domain professionals when a change affects sensitive-data handling, consequential permissions, regulated workflows, retention duties or remediation of external actions. This operating method organizes release evidence; it does not determine those requirements.