MCP in Production: Designing the Tool Layer for Enterprise AI Agents on AWS

The first enterprise AI agent is usually easy to explain. It has one job, a short system prompt, and access to two or three APIs. The tenth agent is where the architecture changes. The hundredth is where an organization discovers that “giving the model tools” was never merely an integration task.

At that scale, customer support, finance, engineering, operations and analytics agents all need to reach overlapping systems. They need Salesforce, Jira, GitHub, payment services, data platforms and internal APIs. Each system has its own authentication model, authorization semantics, failure modes and audit requirements. Every new direct connection creates another place to store credentials, translate schemas, implement retries and explain a tool call to a security reviewer.

The Model Context Protocol (MCP) gives us a standard way to describe and invoke tools. That is important. It removes a large amount of adapter code and creates a shared language between agent applications and tool providers. But the protocol does not, by itself, answer the questions that decide whether an enterprise tool layer is safe to operate:

Which agent may call which tool for this user?
Which parameters are acceptable in this business context?
When must a person approve the action?
How does identity survive the hop from user to agent to downstream system?
How do we keep hundreds of tools out of every model context?
What does a retry mean after a payment, refund or deployment may already have happened?
Can the entire path remain private and observable?

The goal of the tool layer is not merely connectivity. It is governed connectivity.

Related foundation: If you are still deciding whether you need RAG, tools, memory or an agent at all, start with Designing a Production-Grade AI Agent Platform on AWS. This article zooms in on the governed tool layer.

MCP solves the protocol problem, not the operating model

As of August 2026, the current MCP specification is the 28 July 2026 revision. It defines interoperable operations for discovery and invocation, including tool listing and tool calls, with JSON Schema used to describe inputs and outputs. The revision changelog also introduced a stateless interaction model and server/discover, which matters when large systems need to discover capabilities without assuming a long-lived session.

Those are protocol contracts. They are not an enterprise control plane. MCP does not decide whether a finance agent may issue a refund, whether the refund is below a delegated limit, whether the customer is in a restricted jurisdiction, or whether a second approval is required. It cannot know that “delete customer” is materially different from “read customer,” even if both tools are exposed by the same server.

The specification is explicit about several responsibilities that remain with clients and servers. Tool annotations are untrusted unless the client already trusts the server. Clients should show users which tools are exposed and should retain a human in the loop for sensitive operations. Servers should validate inputs, enforce access controls, rate-limit calls and sanitize outputs. Clients should validate results, use timeouts and log tool use.

I treat those statements as an architectural boundary. MCP standardizes the wire contract. The enterprise still owns identity, policy, network placement, reliability, observability and approval semantics.

Why direct agent-to-tool integration breaks down

The fastest path for a proof of concept is often a direct path: the support agent owns a Salesforce client, the developer agent owns a GitHub client, and the finance agent owns a payment client. Each agent carries its own credentials and retry behavior. It works until different teams need the same system or the same agent needs a dozen systems.

Figure 1 — Direct integrations multiply credentials, schemas and failure handling across agent teams.

The coupling appears in five places.

Credentials spread with the topology

If four agents call five systems directly, credential ownership becomes a matrix. Rotation becomes coordinated application work. A compromised agent may inherit every secret embedded for every downstream integration. Even when each agent uses an IAM role rather than a static key, the permission boundary is still distributed across many runtimes.

Schemas drift independently

One team calls a parameter customer_id; another uses accountId. One wrapper returns a typed error; another returns a string. Version upgrades become agent-by-agent migrations. The model sees inconsistent affordances for the same business capability.

Authorization gets confused with availability

A tool being visible to a model does not mean the current user or agent may use it. A model selecting issue_refund is a reasoning result. It is not proof that the caller may refund this customer for this amount.

The model choosing a tool is a reasoning decision. It is not an authorization decision.

Retries acquire business consequences

A generic HTTP client may retry a timeout. A refund service cannot assume that a timeout means nothing happened. Without an idempotency key and a durable operation record, the retry may create a second side effect.

Audit evidence fragments

Application logs show model intent, API logs show a downstream call, and identity logs show a token exchange. If those records do not share correlation identifiers and a consistent actor model, incident review becomes reconstruction rather than observation.

A governed enterprise tool plane on AWS

The architecture I prefer introduces a tool plane between agent applications and enterprise systems. On AWS, Amazon Bedrock AgentCore Gateway can be the managed entry point for MCP tools, while AgentCore Identity, AgentCore Policy, IAM, Secrets Manager, CloudWatch and CloudTrail provide the surrounding controls.

Figure 2 — A shared tool plane separates agent reasoning from access, identity, policy and downstream execution.

The path is intentionally layered:

The user authenticates through Amazon Cognito or an enterprise identity provider.
The agent runs in AgentCore Runtime or another controlled compute environment and receives a user-bound identity context.
The agent uses MCP to discover or invoke a capability through AgentCore Gateway.
The gateway authenticates the inbound caller, invokes policy, applies interceptors or rate limits, and selects the target.
AgentCore Identity or IAM supplies the appropriate outbound identity. Secrets Manager is used only where a downstream credential cannot be represented as an AWS role or delegated token.
The target may be an MCP server, Lambda function, API Gateway REST API, OpenAPI-defined service, Smithy model or supported connector.
Metrics, traces, policy decisions and audit events flow to the observability plane.

This is not a claim that every enterprise must have one global gateway. A single global tool plane can create its own blast radius and ownership bottleneck. I usually segment gateways by trust boundary, business domain, environment or regulatory scope, while standardizing the identity and policy patterns across them.

MCP server versus AgentCore Gateway

An MCP server and a gateway solve different problems.

Concern	MCP server	AgentCore Gateway
Primary responsibility	Implements domain tools and resources	Exposes, aggregates and governs targets
Business semantics	Owns validation and side-effect rules	Applies cross-cutting access controls
Inbound access	Server-specific	JWT, IAM or offloaded patterns
Outbound access	Server-specific credentials	IAM, OAuth, OBO, token passthrough or API keys
Policy	Application logic	Central policy engine plus interceptors
Catalog	Its own tools	Multiple synchronized or dynamic targets
Observability	Domain-level execution details	Gateway metrics, logs, traces and CloudTrail

The gateway does not remove the need for well-designed MCP servers. It gives them a governed front door.

I keep domain rules close to the domain. The refund service should still reject an invalid currency, enforce its idempotency contract and record the financial transaction. Gateway policy should decide whether this actor, agent, tool and parameter combination is allowed to reach that service. Interceptors can normalize or redact. They should not become a second, hidden implementation of the business system.

Identity propagation: authentication is not authorization

Enterprise agent systems have at least three identities in play: the human or workload requesting an outcome, the agent application performing reasoning, and the execution identity presented to the downstream tool. Flattening them into one service role makes access convenient but destroys attribution.

Figure 3 — Authentication, agent permission, tool authorization and human approval are separate decisions.

AgentCore Gateway supports several inbound authorization modes. JWT authorization is useful when an enterprise identity provider issues tokens containing claims that policy can evaluate. IAM authorization is appropriate for AWS-native workloads that can sign requests. Offloaded modes allow an external control to perform authorization while the gateway still applies other controls.

Outbound access is equally important. AgentCore Gateway supports IAM service roles, caller IAM with forward access sessions, OAuth client credentials, authorization code grants, token exchange, token passthrough and API keys. The correct choice depends on what the downstream system must know.

Use a service identity when the downstream action is genuinely performed by the agent platform and user attribution is carried separately.
Use delegated user identity when the downstream system must enforce the user’s own permissions or retain user-level audit evidence.
Use on-behalf-of token exchange when the downstream service should receive a token that represents both the user and the acting agent. AgentCore Identity supports standards-based token exchange for this pattern.
Use token passthrough carefully because it couples trust domains and can overexpose a bearer token. Audience, scope and lifetime still matter.

The policy decision should see enough context to distinguish the cases. A useful authorization tuple is: principal, agent, tool, action, resource, parameters, environment and risk. AgentCore Policy uses Cedar and evaluates gateway requests with default-deny behavior in enforce mode. Forbid rules win over permits, which is the right bias for enterprise controls.

Consider a support agent with a refund tool. A simple policy can permit reading an order, permit address correction for an authenticated support user, permit refunds below $1,000, require approval above that threshold, and always deny customer deletion. The model may propose any of those tools. The policy layer decides which path is executable.

Approval is not a special form of authentication. It is a business control. A human approval record should include the proposed tool, normalized parameters, policy reason, actor, agent, expiry and correlation identifier. If the parameters change after approval, the approval should no longer be valid.

Tool discovery without context-window collapse

A large enterprise catalog can contain hundreds or thousands of tools. Sending every schema to every model on every turn is expensive and counterproductive. The model spends tokens comparing irrelevant options, tool names collide, and prompt-injection opportunities increase with every unnecessary description.

Figure 4 — Discover a small, relevant tool surface instead of putting the full enterprise catalog in every model context.

AgentCore Gateway supports synchronized MCP targets and dynamic discovery. In the default synchronized mode, the gateway indexes target capabilities so that they can participate in its catalog and semantic search. Updates to target definitions require synchronization. Dynamic mode queries a target at request time, which is useful when capabilities change frequently, but it is not interoperable with semantic search or outbound three-legged OAuth today.

The built-in semantic search tool, x_amz_bedrock_agentcore_search, lets an agent query for relevant capabilities before loading full definitions. I combine that with business-domain segmentation:

Support agents search customer and billing domains.
Finance agents search billing and analytics domains.
Developer agents search DevOps and analytics domains.

The discovery result should be the smallest useful surface: a handful of tool names, descriptions and schemas that match the user’s current intent and authorization context. Discovery itself may need authorization. A caller should not learn that a restricted administrative tool exists merely because semantic search found it.

Tool metadata becomes part of the production contract. Names should be stable, descriptions should state the business effect, input schemas should be narrow, and output schemas should be explicit. Version tool contracts as deliberately as APIs. A silent description change can alter model selection behavior even when the HTTP implementation is unchanged.

Private connectivity without pretending the gateway lives in your VPC

Private networking is often described imprecisely in AI architecture diagrams. AgentCore Gateway is an AWS-managed service. It does not move inside a customer VPC. The private design is built from private service endpoints and controlled egress paths.

Figure 5 — PrivateLink protects the path into AgentCore; VPC Lattice provides private gateway egress to tool endpoints.

For an agent running in a private subnet, AWS PrivateLink interface endpoints provide private access to the AgentCore data plane, control plane and gateway endpoints. The agent can call the managed service without traversing the public internet. DNS and endpoint policies should be part of the design, not post-deployment hardening.

For gateway egress to private MCP servers or OpenAPI targets, AgentCore Gateway integrates with VPC Lattice private endpoints. The target remains in a tool VPC while the gateway remains AWS-managed. A regional API Gateway target can also reach VPC resources through VPC Link. Current service limits matter: private endpoint support is not identical for every target type, and Smithy private endpoints are not supported at the time of writing.

Hybrid tools can sit behind private connectivity extended with AWS Direct Connect or a site-to-site VPN. The architectural requirement is not “no public IP” as a slogan. It is a verified path with route ownership, DNS resolution, security-group rules, service authorization and failure monitoring.

Reliability is a tool contract

Agents make reliability harder because a single user goal may produce several tool calls, branches and retries. A partially completed plan can be more dangerous than a clean failure. The platform must distinguish read operations, naturally idempotent writes, writes protected by an idempotency key, and writes that require explicit reconciliation.

Figure 6 — Rate limiting, policy, approval, idempotency and telemetry belong on the tool execution path.

Retries are a business-semantics problem, not just an HTTP problem.

Every side-effecting tool should define:

An idempotency-key contract and its retention period.
Whether the operation is synchronous, asynchronous or eventually consistent.
Which errors are safe to retry and which require reconciliation.
A status lookup that can resolve an ambiguous timeout.
A maximum execution time smaller than the caller’s total deadline.
A compensating action where the business process supports one.

Use Step Functions when a business action has durable state, approval, timeout or compensation. Use Amazon SQS to absorb bursts and decouple long-running work. Send terminally failed messages to a dead-letter queue with enough context for replay or investigation. Do not replay a failed business action merely because a queue makes replay convenient.

AgentCore Gateway supports customer-defined rate limits using dimensions such as target name, tool name and JWT claims. The stricter of service and customer limits applies. Rate limiting protects capacity and cost, but it is not an authorization boundary. The documented implementation can fail open on an internal rate-limit error, so a security requirement must be expressed in policy or in the target service, not only as a throttle.

Interceptors are useful for correlation identifiers, parameter normalization, data-loss prevention and output redaction. Make them idempotent: the gateway may retry an interceptor. Keep them small and observable. An interceptor timeout should not become an unexplained tool timeout.

Observability must join reasoning to execution

A production trace should answer one question without a manual join across five consoles: who asked which agent to perform what business action, which tool was selected, why policy allowed or denied it, which identity reached the target, what the target returned, and whether a retry or approval changed the outcome?

AgentCore Gateway publishes CloudWatch metrics including invocations, throttles, system errors, user errors, latency, duration and target execution time. Gateway logs can flow to CloudWatch Logs, Amazon S3 or Firehose. OpenTelemetry spans are available through the AWS transaction-search path. CloudTrail records Gateway API activity, including management and data events such as InvokeGateway.

I add business dimensions above those service signals:

User or workload subject, with privacy-preserving identifiers.
Agent identifier and deployed version.
Gateway, target, tool and contract version.
Policy decision, matched rule and policy version.
Approval identifier and approver when present.
Idempotency key, correlation identifier and trace identifier.
Normalized outcome: succeeded, denied, throttled, timed out, ambiguous, compensated or failed.

Do not log raw prompts, credentials or unrestricted tool payloads by default. Observability data can become a second copy of sensitive customer information. Redact deliberately, encrypt with KMS-managed keys, apply retention policies and restrict access to the telemetry itself.

Tool output is an untrusted input channel

Tool output should be treated as untrusted input.

An MCP tool can return data that contains instructions, markup, URLs or text crafted to influence the model. A compromised content source can turn an apparently harmless “read ticket” tool into a prompt-injection channel. Trust in the server’s network identity does not make every record returned by that server trustworthy.

Defenses belong at several layers. Use structured output schemas instead of free-form text where practical. Separate data fields from instructions. Limit the amount of returned content. Sanitize active markup. Do not automatically follow URLs or execute commands found in tool results. Use response interceptors for redaction or normalization. Most importantly, re-run authorization before a consequential action even when an earlier read suggested that action.

A tool result can inform reasoning. It cannot grant permission.

A production walk-through: issuing a customer refund

Consider a support agent asked to refund an annual subscription.

The customer-support user signs in through the enterprise identity provider. The agent receives a user-bound JWT with audience, subject and role claims.
The agent queries the gateway catalog for tools related to refunds and subscriptions. Semantic search returns a small set from the customer and billing domains.
The model selects get_subscription. Gateway policy permits the read. The downstream response is validated against its output schema and treated as untrusted data.
The model proposes issue_refund with subscription ID, amount, currency and reason. The agent adds a stable idempotency key derived from the business operation, not from a transient request ID.
Gateway policy evaluates the user, agent, tool and parameters. A refund below $1,000 is permitted for this role; a larger amount is routed to a Step Functions approval workflow.
AgentCore Identity exchanges the inbound identity for a scoped downstream token representing the user and acting agent. The billing system performs its own domain validation.
If the billing call times out, the workflow checks the operation status using the idempotency key before deciding whether a retry is safe.
CloudWatch receives service metrics and traces; CloudTrail records gateway activity; the policy engine records its decision. All records share the same correlation identifier.
The agent tells the user what actually happened. An ambiguous result is reported as pending investigation, not as success and not as an invitation to retry blindly.

This sequence is longer than a direct API call. That is the point. The extra steps make identity, authorization, approval, side effects and evidence explicit. They turn an impressive demo into an operable business process.

Design principles I would take into a review

Separate reasoning from authorization. The model proposes; deterministic controls decide what may execute.
Preserve user and agent identity. Do not collapse every action into one platform role unless the business action truly belongs to the platform.
Keep domain semantics in the domain service. The gateway governs access; the target owns its invariants and side effects.
Discover the smallest relevant tool surface. Catalog size should not become context size.
Default deny and make exceptions visible. Policy versions and matched rules should be auditable.
Design idempotency before retries. Every consequential write needs a business operation identity.
Model private connectivity precisely. Managed services remain managed; PrivateLink and VPC Lattice define the private paths.
Treat approvals as expiring, parameter-bound decisions. Approval is not a reusable bypass.
Treat tool output as untrusted. Validate, minimize, sanitize and re-authorize before action.
Join telemetry end to end. A tool call without actor, policy, idempotency and outcome context is not a complete audit record.

MCP is the right foundation for a portable tool contract. In production, the value comes from the architecture around that contract: a governed catalog, delegated identity, deterministic policy, private reachability, business-aware reliability, human approval and evidence that can survive an incident review.

That is the difference between connecting a model to an API and designing the tool layer for enterprise AI agents.