Design a Bounded AI Assistant with Tools, Permissions, and Context Limits

An AI assistant is bounded only when its executable service contract limits what it can read, call, change, and represent as complete. A prompt that says “do not send” is not a meaningful boundary if the model still holds a sending tool and a credential that permits delivery. Treat each visible capability—searching, summarizing, drafting, updating, sending, deleting, or approving—as a separate controlled operation. Give each one its own information envelope, acting identity, tool, resource permission, action ceiling, evidence record, tests, metrics, and accountable owner.

What should the assistant be allowed to do?

The team should begin with a one-sentence service charter, then break it into specific capabilities and explicit non-goals. A useful pattern is: “For [eligible users], the assistant may [allowed task family] using [approved information scope] to produce [permitted outcome], but it may not [explicit non-goals or consequential decisions].” Before connecting tools, name the deployment setting, supported tasks, knowledge limits, human oversight, risk owner, and prohibited outcomes. These elements reflect NIST AI RMF concerns, while the capability-level format is practical editorial synthesis rather than a NIST or OWASP requirement.

• Replace “help with support” with operations such as find an eligible case, summarize it, create a reply draft, or send an approved reply.
• Replace “manage records” with distinct read, create, update, delete, and approval operations.
• Keep any operation absent when its users, information, authority, evidence, tests, or owner cannot yet be specified.
• Prefer the narrowest tool functionality that delivers the intended outcome.

What information may each capability use?

A context limit should be an information envelope, not merely a token budget. For each capability, specify eligible systems, record types, classifications, object filters, date ranges, freshness expectations, user entitlements, and prohibited data. A larger context window creates neither authority nor truth. When required evidence is inaccessible, missing, or stale, the assistant should decline the unsupported conclusion or qualify its answer. Retrieved documents, attachments, external messages, and API responses remain untrusted content; they must not silently become service instructions merely because retrieval placed them beside the prompt.

• Separate current-session history from persistent memory and state what each may contain.
• Validate information before persistence and isolate memory by eligible user and session.
• Set service-specific expiry, size, integrity, deletion, and sensitive-data rules instead of adopting universal defaults.
• Exclude credentials, secrets, unrelated records, and any data the capability does not need.
• Preserve the source and freshness signals needed to explain an answer or refuse one.

How should identity, tools, and permissions enforce the boundary?

Authentication, authorization, and approval must remain separate decisions enforced in the execution path. Authentication establishes who the user, client, agent, or workload is. Authorization determines whether that identity may perform a named operation on a protected resource. Approval accepts one inspectable proposed action; it does not widen standing permission. Choose deliberately between delegated user authority and a controlled workload identity, and never let the assistant silently inherit a privileged operator account. Tool gateways and downstream systems should validate every operation, resource, object, field, destination, and parameter on the acting identity.

• Expose narrow operations with validated parameters instead of broad mailbox, database, browser, or shell access.
• Use minimum downstream permissions and preserve the user's authorization context when acting for that user.
• Constrain credential duration and intended audience so a token is not portable across unrelated resources.
• Apply service-specific rate, retry, chain-depth, batch, spend, time, idempotency, rollback, and circuit-breaker limits.
• Treat prompts and natural-language guardrails as behavioral inputs, never as authorization controls.

A conversation may feel continuous, but its authority should be divided into small, independently enforced capabilities.

How much action should a capability be able to take?

Every capability needs an explicit action ceiling, with stronger independent controls as it approaches a consequential external effect. The following ladder is practical editorial synthesis, not a universal tiering standard. Keep draft creation separate from sending, proposals separate from accountable decisions, and reversible field updates separate from destructive changes. For a consequential action, show an inspectable preview and bind approval to the actor, tool, target resource, normalized parameters, time, and expiry. Revalidate that exact action immediately before execution; changed parameters require fresh review.

1. Answer or summarize eligible information without changing external state.
2. Recommend a next step or prepare an inspectable proposal without executing it.
3. Create an editable artifact in a restricted, non-final draft state.
4. Perform a narrowly scoped, reversible write with validation, idempotency or rollback, and an audit event.
5. Cause a consequential external action only through valid authorization, action-bound approval, execution policy, and an incident path.
6. Deny a prohibited decision even if a user asks for or appears to approve it.

Approval never supplies missing authorization or converts a prohibited decision into an allowed one. Payments, access grants, destructive operations, production changes, high-stakes professional judgments, and material external commitments belong under qualified-human accountability and deterministic policy appropriate to the organization. The assistant may prepare evidence or a draft when that bounded assistance is safe, but it should not become the decision maker merely because the interaction includes an approval button.

What should happen when the assistant reaches a boundary?

Refusal, safe partial help, human handoff, and security escalation should be explicit service outcomes with genuine stop conditions. The assistant should state the boundary plainly without disclosing sensitive policy details, and it must never claim that a source check, tool call, approval, or write succeeded when it did not. It can offer the safe part of the task—a draft, checklist, or request for missing information—while execution remains stopped. Model confidence alone should never decide whether to approve, refuse, or escalate.

• Use clear reasons: task out of scope, information ineligible, authorization insufficient, approval required, evidence missing or stale, specialist judgment required, dependency unavailable, operational limit reached, or security signal detected.
• Package the original goal, relevant non-sensitive context, attempted capability, reason, available or missing evidence, proposed next step, and trace identifier.
• Route routine user handoffs, business approvals, and security incidents to their distinct accountable destinations.
• Require fresh authorization and approval when a pending action's target, content, parameters, or operating conditions change.

Security signals such as suspected privilege escalation, data exfiltration, memory poisoning, approval bypass, or recursive tool abuse require the relevant incident path, not an ordinary customer-service transfer. The responsible owner needs authority to pause execution, revoke access, preserve appropriate evidence, and coordinate remediation. A routine missing-policy question can instead go to the content or business owner. The evidence package may share a common structure, but urgency, confidentiality, and decision rights differ.

How can teams turn the boundaries into an operational design?

Teams should complete one capability-design canvas row for each user-visible operation and connect every row to enforceable controls, evidence, evaluation, operating metrics, and ownership. Record the eligible actor and authentication, information envelope, session and memory rules, narrow tool operation, acting identity, resource scope, action ceiling, approval, operational limits, refusal, evidence record, test scenarios, metrics, and owner. Avoid a row such as “assistant can access the customer platform.” Name a bounded operation such as “summarize a case already visible to this staff member.”

The three rows may share a conversational interface without sharing data grants, tools, ceilings, tests, metrics, or handoffs. Logs should answer operational questions rather than retain everything: reconstruct who requested what, which capability and policy applied, which evidence and tool classes were used, what authorization and approval occurred, what action resulted, and which versions were active. Redact secrets and limit sensitive content according to privacy, security, and records requirements.

What evidence is needed to release and keep operating the assistant?

Release requires evidence that permitted service and expected denials both work under conditions resembling deployment. Test routine success alongside cross-account access, unauthorized tools, prohibited data, stale evidence, poisoned retrieved content, approval bypass, changed parameters, duplicate retries, dependency failure, exfiltration attempts, and runaway chains. A passing test set is not permanent assurance. Assign owners for service behavior, access grants, business handoffs, and security incidents, and give them authority to pause, revise, or retire an affected capability.

• Reconstruct the requester, capability, policy, evidence class, tool call, authorization result, approval identifier, outcome, and active versions without logging secrets.
• Monitor unexpected tool use, repeated denials, authorization failures, approval changes, anomalous action sequences, drift, latency, resource use, and operational failures.
• Set metrics and alerting rules according to service risk, privacy, records obligations, and credible failure impact.
• Communicate known limitations and failure modes to users and operators before release.
• Review the affected permissions, logs, tests, handoffs, and stop conditions after material change.

Reopen relevant release evidence when models, prompts, retrieval, memory, tools, permissions, policies, data, providers, or operating context materially change. Begin with the smallest useful capability and expand authority through reviewed changes, not prompt edits alone. Involve security, identity, privacy, records, risk, and service owners when sensitive information, persistent memory, privileged access, external commitments, destructive changes, or incident response enter scope. Route legal, regulatory, and other high-stakes professional judgments to qualified specialists and a separately governed process.