Designing a Bounded AI Assistant with Tools, Permissions and Context Limits

A bounded AI assistant is an executable service contract, not a prompt containing a polite list of prohibitions. Telling a model not to send a message is beside the point if the surrounding service still gives it a send tool and a credential that can reach every recipient. The real boundary sits in the information supplied, the operations exposed, the identity used, the permissions checked and the actions the execution path will accept. Design those controls capability by capability, and a conversational interface can remain useful without becoming a bundle of unrelated authority.

What should the assistant be allowed to do?

The team should begin with a one-sentence service charter, then split it into specific capabilities and explicit non-goals. A useful form is: “For [eligible users], the assistant may [allowed task family] using [approved information scope] to produce [permitted outcome], but it may not [explicit non-goals or consequential decisions].” Record the intended users, deployment setting, supported tasks, knowledge limits, human oversight and risk owner before connecting a tool. These concerns align with NIST AI RMF outcomes, although this capability method and template are practical editorial synthesis, not NIST requirements.

Broad verbs hide materially different powers. “Help with customer service” might mean searching approved articles, summarising an eligible case, recommending a response, creating a draft, updating a record or sending a message. Give each operation its own boundary. OWASP’s minimum-function guidance supports exposing a narrow operation when it can replace an open-ended extension, reducing the range of actions available if the model is confused or manipulated without claiming to remove every security or operational risk.

• Name the eligible user and how eligibility is established.
• Describe the exact output or state change that counts as success.
• Separate allowed, proposal-only, approval-gated and prohibited outcomes.
• Assign owners for the service, access grants, operational risk and human handoff.

What information may each capability use?

A context limit should be an information envelope defining eligible data and trust boundaries, not merely a model token budget. For each capability, specify approved systems, record types, classifications, object filters, relevant dates, freshness expectations, user entitlements and prohibited data. NIST’s Map outcomes address knowledge limits, targeted application scope and oversight of output use. A larger context window changes capacity, but it neither creates authority nor makes unsupported material true.

Treat retrieved documents, attachments, external messages and API responses as untrusted content rather than service instructions. Their contents may inform an answer only within the declared task and policy. Session history and persistent memory also need different rules: define what is eligible, how users and sessions are isolated, what validation occurs before persistence, when information expires, how deletion works and what must never be stored. OWASP covers these memory controls but supplies no universal size or retention setting.

• Refuse when the required source or record is outside the user’s entitlement.
• Qualify the answer when essential evidence is missing, inaccessible or stale.
• Prevent retrieved text from silently overriding service instructions.
• Keep secrets, credentials and prohibited sensitive data out of prompts and memory.

How should identity, tools and permissions enforce the boundary?

Authentication, authorisation and approval must remain separate decisions, enforced by tool gateways and downstream systems rather than entrusted to the model. Authentication establishes which user, client or workload is present. Authorisation decides whether that actor may perform a named operation on a protected resource. Approval accepts one particular proposed action. The MCP authorisation specification illustrates distinct client, resource and authorisation-server roles, least-privilege scopes and resource-bound tokens for protected MCP integrations, but it is not a universal architecture for every tool.

Choose deliberately between delegated user authority and a controlled workload identity. Do not let an assistant silently inherit a privileged operator account. Expose operations such as “read eligible case” or “create reply draft”, with validated parameters, instead of broad database, mailbox, browser or shell access. Constrain verbs, resources, objects, fields, destinations, credential duration and token audience in the execution path. OWASP recommends minimum downstream permission, preservation of user authorisation context and complete mediation for requests.

• Apply service-specific limits for rate, retries, chain depth, batch size, spend and elapsed time.
• Use idempotency, rollback and circuit breakers where the operation needs them.
• Keep credentials out of conversational context and persistent memory.
• Treat prompts and natural-language guardrails as behaviour guidance, never as authorisation controls.

A conversation may feel continuous, but its authority should be divided into small, independently enforced capabilities.

How much action should a capability be able to take?

Every capability needs an explicit action ceiling, with stronger independent controls as it moves from answering towards changing external state. A practical editorial ladder is: answer or summarise; recommend or propose; create a draft; perform a bounded reversible write; cause a consequential external action; or encounter a prohibited decision. Keep draft creation separate from sending, reversible field updates separate from destructive changes, and proposals separate from decisions for which a person or governed process remains accountable.

For a consequential action, show the human an inspectable preview and bind approval to the actor, tool, target resource, normalised parameters, time and expiry. Validate that package immediately before execution. OWASP supports separating decision from execution and combining high-impact approval with downstream mediation. Approval never supplies missing authorisation, widens standing permission or converts a prohibited decision into an allowed one. If the recipient, content, target or other material parameter changes, seek fresh validation rather than reusing the earlier approval.

• Keep payments, access grants, destructive operations and production changes under suitable qualified-human and deterministic-policy control.
• Route high-stakes professional judgements and material external commitments through a separately governed process.
• Use external fail-safes where credible impact warrants them, consistent with NCSC secure-design guidance.
• Set operational thresholds from the service’s risk, reversibility and workload rather than borrowing universal numbers.

What should happen when the assistant reaches a boundary?

Refusal, safe partial help, human handoff and security escalation should be explicit service outcomes with genuine stop conditions. Use clear reason categories: task outside scope, information ineligible, insufficient authorisation, approval required, evidence missing or stale, specialist judgement required, dependency unavailable, operational limit reached or security signal detected. State the boundary plainly without revealing sensitive policy detail, and never claim that a source check, tool call, approval or write succeeded when it did not.

Offer only the safe portion of the request, perhaps a draft, checklist or request for missing information. A handoff should carry the original goal, relevant non-sensitive context, attempted capability, reason, available or missing evidence, proposed next step and trace identifier. Keep a routine service handoff, business approval and security incident escalation distinct because their owners and urgency differ. NCSC incident guidance calls for response, escalation and remediation scenarios, trained responders and strong audit logs.

• Stop pending execution while the responsible owner acts.
• Require fresh validation if a proposed action changes.
• Escalate security signals such as approval bypass, privilege escalation, data exfiltration, memory poisoning or recursive tool abuse.
• Test safe failure under conditions resembling the deployed service.

How can teams turn these boundaries into an operational design?

Teams should complete one capability-design canvas row for every user-visible operation and connect it to enforceable controls, evidence, tests, metrics and ownership. Record the eligible actor, authentication, information envelope, session and memory rules, narrow tool, acting identity, resource scope, action ceiling, approval, operational limits, refusal behaviour, evidence record, evaluation scenarios and accountable owner. The canvas is practical editorial synthesis informed by NIST and OWASP; it is useful only when each field maps to the running service rather than remaining a documentation exercise.

Consider an internal support assistant used by account-service staff. Finding a case, drafting a reply and sending an approved response may share one conversational surface, yet OWASP’s minimum-function and minimum-permission principles support separate controls. Evidence should reconstruct the applicable capability, policy, sources, tool calls, authorisation outcome, approval identifier and result while redacting sensitive content. NIST governance outcomes also support clear roles, differentiated human and AI responsibilities, monitoring and periodic review.

What evidence is needed to release and keep operating the assistant?

Release requires evidence that permitted service and expected denials both work in deployment-like conditions, followed by monitoring and reassessment when the service changes. Test ordinary tasks alongside cross-account requests, unauthorised tools, stale evidence, poisoned retrieved content, prohibited data, approval bypass, changed parameters, duplicate retries, dependency failure, exfiltration attempts and runaway chains. NIST calls for deployment-representative testing, safe failure, documented limits, production monitoring and regular safety and security evaluation; NCSC also recommends appropriate security evaluation before release and communication of known limitations.

Logs should make it possible to reconstruct who requested what, which capability and policy applied, which source and tool classes were used, which authorisation and approval decisions occurred, what action resulted and which versions were active. Do not retain secrets or unbounded sensitive context simply because it might be useful later. Choose content, access and retention according to the organisation’s privacy, security and records requirements.

• Monitor unexpected tool use, repeated denials, authorisation failures, approval changes, anomalous action sequences, latency, resource use and operational failures.
• Give service, access, business-handoff and security owners authority to pause, revise or retire the relevant capability.
• Reopen affected tests after material changes to models, prompts, retrieval, memory, tools, permissions, policies, data, providers or operating context.

Begin with the smallest useful capability and expand authority through reviewed changes, not prompt edits alone. Consult the organisation’s security, identity, privacy, records, risk and service owners when a capability touches sensitive information, persistent memory, privileged access, destructive change, external commitments or incident response. Route legal, regulatory and other high-stakes professional judgements to qualified specialists and a separately governed process. The practical release question is not whether the assistant sounds capable, but whether its useful actions, denials, evidence and stop conditions remain observable and enforceable.