Practical intelligence for accountable AI programmes.
A practical method for defining an AI assistant’s information boundaries, tool permissions, approvals, refusals and accountable escalation paths.
ModelFold Editorial Team9 min read
A bounded AI assistant is an executable business service contract, not a system prompt containing polite prohibitions. If the service holds a credential and can call a send tool, telling the model not to send is not the real boundary. The boundary lies in the information the service can retrieve, the narrow operations it can invoke, the identity used for each call, the permissions enforced downstream and the conditions that stop execution. Design those controls separately for every user-visible capability so a routine request cannot inherit unrelated data or authority.
The control principles to carry forward
Start with a one-sentence service charter, then split it into named operations. A useful form is: For eligible users, the assistant may perform an allowed task family using an approved information scope to produce a permitted outcome, but it may not perform stated non-goals or consequential decisions. Name the users, deployment setting, oversight and risk owner before connecting a tool. NIST identifies intended purpose, supported tasks, knowledge limits, application scope, oversight and risk tolerance as matters to document.
Each verb deserves its own boundary because it carries different authority. Searching a case requires entitled read access; drafting needs an approved workspace; sending creates an external effect. OWASP recommends minimum tool functionality and a narrow operation where it can replace open-ended access. The capability method is practical editorial synthesis informed by NIST and OWASP guidance, not a requirement imposed by either body. Its value is that reviewers can inspect a concrete service promise rather than debate what a broad word such as “manage” might mean.
A context limit should be an information envelope defining eligible data and trust boundaries, not merely the model’s token capacity. For every capability, record approved systems, record types, classifications, object filters, relevant dates, freshness expectations and the user entitlements that must still hold. Also name prohibited data. NIST’s Map outcomes support documenting knowledge limits, application scope and how outputs may be used and overseen; a bigger context window does not create authority or make unsupported information true.
Treat retrieved documents, attachments, external messages and API responses as untrusted content. OWASP advises separating that content from instructions so a sentence inside a case attachment cannot silently redefine the service. Session history and persistent memory also need separate rules: decide what may persist, validate it before storage, isolate it by user and session, protect its integrity, set service-specific size and expiry limits and support deletion. When required evidence is inaccessible, absent or stale, the assistant should qualify the answer, request what is missing or refuse the unsupported part.
Authentication, authorisation and approval must remain separate decisions, with tool gateways and downstream systems enforcing the boundary. Authentication establishes which user, client or workload is present. Authorisation decides whether that identity may perform a specific operation on a protected resource. Approval accepts one proposed action. For protected MCP integrations, least-privilege scopes and resource-bound authorisation can support this separation, but MCP is one protocol rather than a universal design for every tool.
Do not silently borrow a privileged operator account. OWASP recommends minimum downstream permissions, preservation of the user’s authorisation context and mediation by downstream systems. The execution path should therefore reject an out-of-scope object or verb even when the model requests it confidently. Prompts and natural-language guardrails may shape behaviour, but they are not authorisation controls. NCSC guidance likewise supports least privilege, secure defaults, restrictions on AI-triggered actions and external fail-safes where credible impact calls for them.
A conversation may feel continuous, but its authority should be divided into small, independently enforced capabilities.
Every capability needs an explicit action ceiling, with stronger independent controls as it moves from answering towards changing external state. A practical editorial ladder separates answering or summarising, recommending or proposing, creating a draft, making a bounded reversible write, causing a consequential external action and encountering a prohibited decision. The ladder is not an OWASP or NCSC standard; it translates their minimum-permission, action-restriction and fail-safe principles into a design conversation that product, security and service teams can use together.
Keep draft creation separate from sending, proposals separate from accountable decisions and reversible updates separate from destructive changes. Before a consequential action, show an inspectable preview and bind approval to the actor, tool, target, normalised parameters, time and expiry. Validate authorisation again at execution. Approval cannot supply missing authorisation, enlarge standing permission or make a prohibited decision permissible. Payments, access grants, destructive operations, production changes, material external commitments and high-stakes professional judgements need qualified human and deterministic policy control appropriate to the organisation.
Refusal, safe partial help, human handoff and security escalation should be designed as explicit service outcomes with genuine stop conditions. State the boundary plainly without revealing sensitive policy detail, and never claim that a source check, tool call, approval or write succeeded when it did not. NIST’s Measure outcomes include safe failure beyond documented knowledge limits. The service should therefore be tested to stop cleanly, preserve an honest status and offer only the useful portion that remains within scope.
A handoff should carry the original goal, relevant non-sensitive context, attempted capability, reason, available or missing evidence, proposed next step and a trace identifier. Keep a routine user handoff, business approval and security incident escalation distinct because their owners and urgency differ. NCSC recommends incident plans with escalation, remediation scenarios, trained responders and useful audit logs. Security signals such as approval bypass, privilege escalation, data exfiltration, memory poisoning or recursive tool abuse should halt pending execution and reach the security path; a changed proposal requires fresh validation.
Complete one capability-design canvas row for every user-visible operation, then connect it to enforceable controls, evidence, tests, operating measures and an accountable owner. Record the eligible actor and authentication, information envelope, session and memory rules, narrow tool operation, acting identity, resource scope, action ceiling, approval, operational limits, refusal, evidence record, evaluation scenarios, measures and escalation destination. This canvas is editorial synthesis: it becomes useful only when each field maps to working service behaviour rather than remaining a documentation checklist.
| Capability | Information and tool boundary | Action ceiling and approval | Evidence, tests, metrics and owner |
|---|---|---|---|
| Find and summarise an eligible support case | Use delegated read access only for cases already visible to the staff member; exclude unrelated accounts, credentials and hidden notes. | Answer or summarise only; refuse inaccessible or unsupported records. | Record policy, case and source identifiers and denial reasons. Test cross-account, hidden-note, stale-evidence and injected-attachment cases; the service owner handles exceptions. |
| Create a reply draft | Use the eligible case, approved knowledge and response policy; write only to a non-final workspace, with no send permission. | Create a draft for review; omit unsupported commitments and route missing judgement. | Record source, policy, draft and reviewer identifiers. Test missing support, unauthorised promises, sensitive data and adversarial retrieved instructions; the content or policy owner resolves gaps. |
| Send an approved reply | Use a separate narrow send operation and an identity authorised only for the intended channel and resource. | Execute only when recipient, content reference, authorisation and unexpired approval still match; prevent duplicate execution. | Record recipient, sender, approval, outcome and duplicate-prevention evidence. Test changed parameters, expired approval, missing authority, retries and outages; the messaging owner investigates execution failures. |
Release requires evidence that permitted service and expected denials both work in deployment-like conditions, followed by monitoring and change-triggered reassessment. Test ordinary tasks alongside cross-account access, unauthorised tools, stale evidence, poisoned retrieved content, prohibited data, approval bypass, changed parameters, duplicate retries, dependency failure, data-exfiltration attempts and runaway chains. NIST calls for deployment-representative testing, safe failure, production monitoring and regular safety and security evaluation; NCSC also recommends appropriate security evaluation before release and communication of known limitations.
Logs should reconstruct those questions without retaining secrets or unbounded sensitive context. Monitor capability-level outcomes such as unexpected tool use, repeated denials, authorisation failures, approval changes, anomalous action sequences, drift, latency, resource use and failures according to organisational risk, privacy and records requirements. Assign owners for service behaviour, access grants, business handoffs and security incidents, each with authority to pause, revise or retire the capability. Monitoring is evidence for investigation and improvement, not proof that broad authority has become safe.
Reopen affected tests and release evidence after material changes to models, prompts, retrieval, memory, tools, permissions, policies, data, providers or operating context. NCSC notes that changes to data, models and prompts can alter behaviour, and recommends watching for sudden or gradual security-relevant changes while respecting privacy requirements. Begin with the smallest useful capability, release it only when success and denial are observable, and expand authority through reviewed changes. Bring in security, identity, privacy, records, risk and service owners wherever sensitive information, privileged access, persistent memory, destructive change or incident response is involved.
A bounded AI assistant is a business service whose tasks, information, identities, tools, actions, refusals, evidence and owners are explicitly limited. Those limits are enforced by the surrounding service and downstream systems, not only by instructions given to the model.
Create one row for each user-visible capability rather than one row for the whole assistant. Record the eligible actor, data scope, narrow operation, acting identity, resource permission, action ceiling, approval, limits, evidence, tests, measures and accountable owner.
Define eligible sources and records, user entitlements, object filters, freshness, trust classes, session history, persistent memory and prohibited data. Set size, expiry, retention and deletion rules for the service instead of adopting universal thresholds.
No. Approval accepts one proposed action, but it cannot replace downstream authorisation, reduce excessive standing permission or permit a prohibited decision. The execution path must still validate the identity, resource, operation and approved parameters.
It should refuse or pause when the task, information or authority is out of scope; evidence is missing or stale; specialist judgement is required; a dependency or operational limit prevents safe completion; or a security signal appears. It may offer safe partial help, then route a structured handoff to the responsible business, service or security owner.
This article was researched using the following sources:
We report on how AI actually lands inside a business. Our work starts from named sources, separates what we found from what we think, and uses AI assistance for research and drafting under documented editorial controls. We are not a substitute for individual expert review.
Build a controlled generative AI drafting workflow that keeps approved evidence, generated prose, review decisions and material edits traceable.
A practical method for building reproducible AI evaluation scenarios that cover routine work, difficult boundaries, known failures and prohibited behaviour.
Build a maintainable AI inventory, rate inherent exposure across four clear dimensions, and route each use to proportionate review as its context changes.
