How to Design a Bounded AI Assistant with Tools, Permissions and Context Limits

A bounded AI assistant is an executable business service contract, not a system prompt containing polite prohibitions. Telling a model not to send a message offers little protection if the surrounding service still gives it a sending tool and a credential with broad access. The effective boundary lies in the information the service can retrieve, the operations it exposes, the identity it uses and the controls that accept or deny each action. Designing those elements capability by capability makes the service inspectable before release and manageable when models, tools or business needs change.

What should the assistant be allowed to do?

The team should begin with a one-sentence service charter, then split it into precise user-visible capabilities and explicit non-goals. A useful form is: “For [eligible users], the assistant may [allowed task family] using [approved information scope] to produce [permitted outcome], but it may not [non-goals or consequential decisions].” This forces early agreement about intended users, deployment setting, supported work, knowledge boundaries, oversight and risk ownership. Those subjects reflect outcomes in the NIST AI Risk Management Framework, although the charter itself is a practical editorial method rather than a NIST requirement.

Broad verbs hide materially different powers. Replace “help with customer service” or “manage cases” with operations such as find, summarise, recommend, draft, update, send, delete and approve. A case search needs read access; a reply draft needs a restricted workspace; a send operation creates an external commitment. OWASP recommends exposing minimum tool functionality and choosing a narrow function over an open-ended extension where possible. Each operation should therefore receive its own information scope, identity, permission, action ceiling, tests, metrics and owner.

• Name the eligible users and how their identities will be established.
• State the supported task, permitted output and systems it may use.
• Record prohibited outcomes and decisions that remain with accountable specialists.
• Assign owners for service behaviour, access grants, handoffs and security incidents before tools are connected.

What information may each capability use?

A context limit should be an information envelope defining eligible data and trust boundaries, not merely a model token budget. For every capability, specify approved systems, record types, classifications, object filters, relevant dates, freshness expectations and the user's existing entitlements. Also name prohibited data explicitly. NIST's Map outcomes call for documented knowledge limits, targeted application scope and oversight of output use. A larger context window cannot extend authority, restore missing evidence or make an unsupported statement reliable.

Retrieved documents, attachments, external messages and API responses should enter the envelope as untrusted content, separated from service instructions and validated where appropriate. That OWASP principle matters because a document can contain text that attempts to redirect the assistant or invoke a tool. Separation reduces exposure but does not make retrieved material inherently safe. When a necessary record is inaccessible, missing or stale, the capability should qualify its answer, request suitable information or refuse rather than quietly substituting assumptions.

Define conversational history and persistent memory separately. State what may remain within a session, what may persist afterwards, how users and sessions are isolated, how entries are validated, when they expire and how they can be deleted. OWASP's memory guidance also covers size limits, sensitive-data review and integrity protection. Choose those settings for the service's risk, privacy and records obligations; there is no universal expiry period or memory size that makes persistence safe.

How should identity, tools and permissions enforce the boundary?

Authentication, authorisation and approval must remain separate decisions, enforced through tool gateways and downstream systems rather than entrusted to the model. Authentication establishes who or what is present: a user, client, agent or workload. Authorisation decides whether that identity may perform a defined operation on a protected resource. Approval accepts one inspectable proposed action. The Model Context Protocol specification illustrates this separation through clients, protected resource servers and authorisation servers, but its protocol requirements apply to protected MCP integrations, not every tool architecture.

Choose deliberately between delegated user authority and a controlled workload identity. Never let the assistant silently inherit a privileged operator account. OWASP recommends minimum downstream permissions, preservation of the user's authorisation context and complete mediation by downstream systems. Expose operations such as “read eligible case” or “create reply draft”, with validated parameters, instead of a general mailbox, database, browser or shell. Constrain verbs, resources, objects, fields, destinations, credential duration and intended token audience in the execution path.

Prompts and natural-language guardrails can guide behaviour, but they are not authorisation controls. The NCSC recommends least privilege, secure defaults, restrictions on AI-triggered actions and external fail-safes where appropriate. Apply service-specific limits for rates, retries, chain depth, batches, spending, elapsed time, idempotency, rollback and circuit breaking. These values should follow the operation's impact and workload; copying universal thresholds from an example merely replaces design with guesswork.

A conversation may feel continuous, but its authority should be divided into small, independently enforced capabilities.

How much action should a capability be able to take?

Every capability needs an explicit action ceiling, with stronger independent controls as it moves from answering towards changing external state. A practical editorial ladder is to answer or summarise, recommend or propose, create a draft, make a bounded reversible write, cause a consequential external action, or encounter a prohibited decision. This ladder is not an NCSC or OWASP standard. It translates their principles of restricted actions, minimum functionality, downstream mediation and external fail-safes into a design conversation that product, security and service owners can use together.

• Keep a draft in a non-final workspace, separate from any sending permission.
• Keep a reversible field update separate from deletion or another destructive change.
• Present consequential actions as inspectable previews before independent execution.
• Retain payments, access grants, production changes, destructive operations, material external commitments and high-stakes professional judgements under appropriate qualified-human and deterministic-policy control.
• Remove prohibited decisions from the capability altogether and enforce their denial downstream.

For a consequential action, bind approval to the actor, tool, target resource, normalised parameters, time and expiry, as OWASP advises, then validate that binding immediately before execution. If the recipient, content, amount, destination or other relevant parameter changes, require fresh validation. Approval never supplies missing authorisation, widens standing permission or converts a prohibited decision into an allowed one. The approver should see what will happen, under which identity and with which material effect, rather than approving a vague conversational intention.

What should happen when the assistant reaches a boundary?

Refusal, safe partial help, human handoff and security escalation should be designed as explicit service outcomes with genuine stop conditions. NIST's Measure outcomes include safe failure beyond documented knowledge limits. Useful reason categories include an out-of-scope task, ineligible information, insufficient authorisation, required approval, missing or stale evidence, specialist judgement, an unavailable dependency, an operational limit or a security signal. The response should state the boundary plainly without revealing sensitive policy details or pretending that a source check, tool call, approval or write succeeded.

Offer only the safe portion of the task: perhaps a draft, a checklist or a request for missing information. A handoff package should carry the original goal, relevant non-sensitive context, attempted capability, reason, available or missing evidence, proposed next step and trace identifier. Route routine service help, business approval and security incidents to their distinct owners. NCSC guidance calls for incident plans covering response, escalation and remediation scenarios, trained responders and high-quality audit logs; that incident path is different from an ordinary user handoff.

Execution must stop while the responsible owner acts. If the proposed action changes during review, the prior approval must not silently resume it. Security-oriented tests should include abuse cases identified by OWASP, such as approval bypass, privilege escalation, data exfiltration, memory poisoning and recursive tool use. These are starting points for a deployment-specific threat model, not an exhaustive list. A refusal that sounds correct but leaves a queued action running is not a successful boundary.

How can teams turn the boundaries into an operational design?

Teams should complete one capability-design canvas row for every user-visible operation and connect it to enforceable controls, evidence, tests, operating metrics and ownership. Record the eligible actor and authentication, information envelope, session and memory rules, narrow tool operation, acting identity, resource scope, action ceiling, approval, operational limits, refusal behaviour and escalation destination. Then add the evidence record, evaluation scenarios, production signals and owner with authority to pause, revise or retire the capability. The canvas is practical editorial synthesis supported by NIST and OWASP control principles, not a prescribed framework.

The shared conversation does not justify shared authority. OWASP's minimum-functionality and minimum-permission guidance supports separating reading, drafting and sending, while its monitoring guidance covers decisions, tool calls, authorisation outcomes, approval identifiers, policy versions, results, resource use and anomalous patterns with sensitive data redacted. NIST governance outcomes also call for clear roles, communication, differentiated human and AI responsibilities, ongoing monitoring and periodic review. Logging should answer operational questions without retaining secrets or unlimited raw context.

What evidence is needed to release and keep operating the assistant?

Release requires evidence that permitted service and expected denials both work under deployment-like conditions, followed by monitoring and change-triggered reassessment. NIST AI RMF outcomes include representative testing, production monitoring, documented generalisation limits, safe failure and regular safety and security evaluation. NCSC guidance likewise recommends appropriate security evaluation before release and communication of known limitations and failure modes. Test routine success alongside cross-account requests, unauthorised tools, stale evidence, poisoned retrieval, prohibited data, approval bypass, changed parameters, duplicate retries, dependency failures, exfiltration attempts and runaway chains.

Logs should reconstruct who requested what, which capability and policy applied, which source and tool classes were used, which authorisation and approval decisions occurred, what action resulted and which versions were active. Redact secrets and limit sensitive content under the organisation's privacy, security and records rules. Monitor unexpected tool use, repeated denials, authorisation failures, approval changes, anomalous action sequences, drift, latency, resource use and failures at capability level. Assign owners able to investigate evidence and pause service rather than leaving dashboards without accountable action.

Reopen affected tests and release evidence after material changes to models, prompts, retrieval, memory, tools, permissions, policies, data, providers or operating context. NCSC specifically notes that changes to data, models and prompts can alter behaviour and should be reflected in evaluation, while monitoring should detect sudden or gradual security-relevant changes without disregarding privacy. Begin with the smallest useful capability and expand authority through reviewed changes, not prompt edits alone. Involve security, identity, privacy, records, risk and service owners whenever the proposed boundary touches their responsibilities.