How to Design a Bounded AI Assistant with Tools, Permissions and Context Limits

A bounded AI assistant is an executable service contract, not a system prompt with a list of prohibitions. Telling a model not to send a message means little if the surrounding service still gives it a send tool and a credential that can reach every recipient. The dependable boundary sits in the information filters, identities, tool gateways, downstream permissions, approval checks and stop conditions around the model. Design those controls separately for each visible capability: searching is not drafting, drafting is not updating, and updating is not sending.

What should the assistant be allowed to do?

Start with a one-sentence service charter, then break it into specific operations and explicit non-goals. A useful form is: “For [eligible users], the assistant may [allowed task family] using [approved information scope] to produce [permitted outcome], but it may not [non-goals or consequential decisions].” The sentence forces the team to name who receives the service, which work is supported, what information is eligible and where accountable human judgement remains.

• Replace “help with customers” with operations such as find a case, summarise eligible records, draft a reply or send an approved response.
• Record the deployment setting, supported tasks, knowledge limits, human oversight, risk owner and prohibited outcomes before connecting tools.
• Keep delete, approve, publish, pay, grant access and production-change operations separate from lower-impact reading or drafting.

NIST's AI Risk Management Framework calls for documenting intended purposes, deployment settings, supported tasks, knowledge limits, application scope, oversight and risk tolerances. OWASP likewise recommends minimum tool functionality and a narrow operation where an open-ended extension is unnecessary. Turning those principles into capability rows is practical editorial synthesis, not a requirement imposed by either body. Its value is operational: reviewers can inspect a precise service promise instead of interpreting a broad verb such as “manage”.

What information may each capability use?

Each capability needs an information envelope, not merely a model token limit. Specify the approved systems, record types, classifications, object filters, date range, freshness expectation and user entitlements that determine what may enter context. Name prohibited data just as clearly. NIST's Map outcomes support documenting knowledge limits, targeted application scope and oversight of output use; a larger context window does not create authority or make unsupported material reliable.

• Classify retrieved documents, external messages, attachments and API responses as untrusted content rather than service instructions.
• Define session history separately from persistent memory, including eligibility, isolation, validation, expiry, deletion and material that must never persist.
• Refuse or qualify the result when required evidence is inaccessible, missing or stale, instead of filling the gap from general model knowledge.

OWASP advises separating untrusted external content from instructions and validating material before it enters context. Its memory guidance also covers validation before persistence, user and session isolation, expiry and size limits, sensitive-data review and integrity protection. Those are design categories, not universal settings. The organisation still has to choose service-specific limits with its privacy, security and records owners, and retrieval or persistent memory should never be treated as proof that an answer is current or authorised.

How should identity, tools and permissions enforce the boundary?

Authentication, authorisation and approval must remain separate decisions. Authentication establishes which user, client, agent or workload is present. Authorisation determines whether that identity may perform a defined operation on a protected resource. Approval accepts one proposed action. A confirmation button cannot repair absent authorisation, and a prompt instruction cannot enforce resource permissions. The tool gateway and downstream system must mediate the request against the acting identity.

• Choose deliberately between delegated user authority and a controlled workload identity; never inherit a privileged operator account silently.
• Expose validated operations such as read an eligible case or create a draft, not broad mailbox, database, browser or shell access.
• Constrain verbs, resources, objects, fields, destinations, credential duration and token audience in the execution path.
• Set service-specific limits for rate, retries, chain depth, batch size, spend, time, idempotency, rollback and circuit breaking.

For protected MCP integrations, the authorisation specification separates clients, protected resource servers and authorisation servers, recommends least-privilege scopes and supports binding tokens to an intended resource. Those protocol rules do not govern every tool architecture, but the separation is instructive. OWASP recommends minimum downstream permissions, preservation of user authorisation context and complete downstream mediation. NCSC guidance also supports least privilege, secure defaults, restricted actions and external fail-safes where appropriate.

A conversation may feel continuous, but its authority should be divided into small, independently enforced capabilities.

How much action should each capability be able to take?

Give every capability an explicit action ceiling, with stronger independent controls as it moves from producing information to changing external state. A practical editorial ladder is: answer or summarise; recommend or propose; create a draft; perform a bounded reversible write; cause a consequential external action; or encounter a prohibited decision. This ladder is a design aid supported by least-privilege and action-restriction principles, not a universal impact classification.

• Keep draft creation separate from sending, proposals separate from accountable decisions and reversible field updates separate from destructive changes.
• Before consequential execution, show an inspectable preview and bind approval to the actor, operation, target, normalised parameters, time and expiry.
• Revalidate authorisation and approval at execution, including after any change to recipient, content, amount, destination or other material parameter.

OWASP advises separating decision from execution for high-impact actions and retaining downstream authorisation checks even when a human approves. Approval therefore never widens standing permission or converts a prohibited decision into an allowed one. Payments, access grants, destructive operations, production changes, material external commitments and high-stakes professional judgements require qualified human accountability and deterministic policy controls appropriate to the organisation. NCSC similarly recommends restricting AI-triggered actions and adding external fail-safes where needed.

What should happen when the assistant reaches a boundary?

Refusal, safe partial help, human handoff and security escalation should be explicit service outcomes with genuine stop conditions. Use plain reason categories: task out of scope, information ineligible, authorisation insufficient, approval required, evidence missing or stale, specialist judgement required, dependency unavailable, operational limit reached or security signal detected. NIST's Measure outcomes support safe failure beyond documented knowledge limits and evaluation in conditions resembling deployment.

• State the boundary without revealing sensitive policy details or claiming that a source check, tool call, approval or write succeeded when it did not.
• Offer only the safe portion, such as a draft, checklist or request for missing information.
• Package the goal, non-sensitive context, attempted capability, reason, evidence status, proposed next step and trace identifier for the responsible owner.

Keep routine user handoff, business approval and security incident escalation distinct. They may share evidence, but they have different destinations, urgency and authority. Execution must remain stopped while the relevant owner acts, with fresh validation if the proposed action changes. NCSC recommends incident plans with response, escalation, remediation scenarios, trained responders and audit logs. OWASP's abuse cases include approval bypass, privilege escalation, data exfiltration, memory poisoning and recursive tool use, all of which warrant tested stop behaviour.

How can teams turn these boundaries into an operational design?

Complete one capability-design canvas row for every user-visible operation, and connect that row to enforceable controls, evidence, tests, metrics and ownership. Record the eligible actor and authentication, information envelope, session and memory rules, narrow tool, acting identity, resource scope, action ceiling, approval, operational limits, refusal behaviour, evidence record, evaluation scenarios and accountable owner. NIST governance outcomes support clear roles, differentiated human and AI responsibilities, ongoing monitoring and periodic review.

• Avoid rows such as “assistant can access the CRM”; name the exact operation and the state or output that counts as success.
• Give read, draft and send operations separate grants even when users experience them through one conversational surface.
• Define who can pause, revise or retire each capability, not merely who maintains its documentation.

Consider an internal support assistant that helps staff find approved case information and prepare customer replies, but does not decide compensation, alter entitlements or make regulatory determinations. The three operations below share an interface while retaining distinct information, tools, permissions, ceilings and handoffs. OWASP's minimum-function and minimum-permission guidance supports that separation, while its monitoring guidance identifies decisions, tool calls, authorisation results, approvals, policy versions, outcomes and anomalous patterns as useful evidence categories.

What evidence is needed to release and keep operating the assistant?

Release requires evidence that permitted service and expected denials both work under deployment-like conditions, followed by monitoring and reassessment after material change. NIST calls for deployment-representative testing, production monitoring, documented generalisation limits, safe failure and regular safety and security evaluation. NCSC likewise recommends appropriate security evaluation before release and communication of known limitations and failure modes. A successful happy-path demonstration is therefore necessary but insufficient.

• Test supported work alongside cross-account access, unauthorised tools, stale evidence, poisoned retrieved content, prohibited data, approval bypass, changed parameters, duplicate retries, dependency failure, exfiltration attempts and runaway chains.
• Monitor unexpected tool use, repeated denials, authorisation failures, approval changes, anomalous action sequences, drift, latency, resource use and failures at capability level.
• Assign owners for service behaviour, access grants, business handoffs and security incidents, each with authority to pause the affected capability.

Logs should reconstruct who requested what, which capability and policy applied, which source and tool classes were used, which authorisation and approval decisions occurred, what action resulted and which versions were active. Do not retain secrets or unbounded sensitive context; logging content, access and retention must follow organisational privacy, security and records requirements. NCSC recommends watching for sudden and gradual security-relevant behaviour changes while respecting privacy and data-protection obligations.

Reopen affected tests and release evidence after material changes to models, prompts, retrieval, memory, tools, permissions, policies, data, providers or operating context. Begin with the smallest useful capability and expand authority through reviewed changes rather than prompt edits alone. Bring in security, identity, privacy, records, risk and service owners when sensitive information, persistent memory, privileged access, destructive change, external commitments or incident response is involved, and route regulated or other high-stakes judgement to qualified specialists through a separately governed process.