How to Connect AI Agents to Internal APIs and Private Business Systems

TL;DR

• Keep the model away from credentials and the network: it proposes a tool call, and trusted server side code makes the actual request to your internal systems.
• Reach private endpoints from the inside out or over private paths: outbound only tunnels, private cloud links such as PrivateLink and Private Service Connect, or VPN subnet routers, with a private MCP server in front.
• A private network is not a security model: authenticate every request, bind tokens to the target service, filter egress, and give each agent only the tools it needs.
• Wrap legacy systems in an anti corruption layer, expose narrow typed tools instead of raw database access, and put human approval in front of anything irreversible.

Most systems an AI agent needs sit behind a firewall: the ERP, the CRM, the billing database, the internal admin API that nobody has documented in years. The quick fix is to expose an endpoint or hand the agent a shared admin key, and that shortcut turns a helpful agent into a liability.

This guide covers internal API integration for AI agents the safer way: how private APIs for AI agents stay private, what AI agent access to internal systems should look like, and how to reach legacy systems and enterprise data without exposing anything sensitive.

The angle is boundaries: where the model, the tool code, and your systems sit relative to each other, and what stands between them. If you are new to tool calling, start with our guide on how AI agents connect to databases and APIs, then come back here for the deployment side.

Where the Trust Boundary Belongs When Agents Access Internal Systems

The trust boundary belongs between the model and everything else. The model proposes a tool call, and trusted server side code, not the model, holds the credentials, opens the network connection, and talks to your internal systems. Every other control in this guide builds on that one rule.

An agent deployment is easiest to reason about as four zones, each with a different level of trust.

The Four Zones: Model, Control Layer, Tool Code, and Protected Core

1. The model (untrusted): a reasoning engine that proposes which tool to call and with which arguments. Its output is a request, never a command. Models can be steered by prompt injection, can hallucinate arguments, and read untrusted content from documents, emails, and tool results.
2. The control layer (governed): the agent runtime and any gateway in front of your tools. It runs the loop, intercepts outbound tool calls, verifies identity, applies policy, logs everything, and pauses risky actions for approval.
3. The tool code (trusted): server side code that receives the model's arguments, validates them against strict schemas, takes the signed in user's identity from the session, and makes the real call. This is the only layer that should touch your network.
4. The protected core (isolated): internal APIs, databases, and legacy systems. They sit in private networks, are reachable only through private paths, and still enforce their own access controls.

Why the Model Should Never Hold Credentials or Network Access

If the model can make requests itself, anyone who can put text in front of it can try to steer those requests. There are four practical reasons to keep it out of the loop:

• Prompt injection: instructions hidden in a document, email, or API response can hijack a model with direct access. When only tool code touches the network, a hijacked model can ask for something, but validated code decides whether it happens.
• Credential leakage: keys and connection strings placed in prompts or model environments can end up in context windows, logs, and chat history.
• Identity: the user's identity must come from the authenticated session, not from an argument the model wrote. A model that can say "act as user 42" can be talked into it.
• Containment: models make mistakes. A trusted wrapper that validates arguments and checks state catches malformed requests before they reach a system of record.

How to Reach Private Endpoints Without Exposing Them

You reach a private endpoint by leaving it private and building the path to it: either the connection starts from inside your network, or it travels over a private route that never touches the public internet. Three patterns cover most cases: outbound only tunnels, private cloud links, and VPNs with subnet routers. In front of whichever you pick, put a private MCP server or gateway so the agent never talks to your backend directly.

Outbound Only Tunnels for On Premises and Firewalled Systems

A small connector running inside your network opens an outbound encrypted connection to a relay, and requests from the agent platform travel back down that connection. There is no inbound firewall rule, no public IP address, and no open port to scan.

Cloudflare Tunnel is a well known example. OpenAI also documents a Secure MCP Tunnel that lets private MCP servers connect to supported OpenAI products without opening inbound firewall ports: a client runs inside your network and needs only outbound HTTPS plus local reachability to the MCP server.

• Best for: systems behind strict firewalls where opening inbound ports is not allowed
• The tradeoff: traffic passes through the provider's relay, so review their data handling terms and treat connector hosts like production infrastructure by patching them and rotating their keys

Private Cloud Links: PrivateLink, Azure Private Link, and Private Service Connect

AWS PrivateLink, Azure Private Link, and Google Cloud Private Service Connect expose a specific service as a private endpoint inside your virtual network. Traffic stays on the provider's backbone over private IP addresses, and access is scoped to that one service rather than your whole network.

Agent platforms are building on this. Google documents agents reaching private endpoints through Service Directory and Private Service Connect, and Vercel's guide to private enterprise agents has tool code reach internal APIs through Secure Compute, AWS PrivateLink, or static IPs while the model never connects to your systems.

• Best for: an agent runtime in the cloud reaching private services in your cloud accounts
• The tradeoff: you are tied to a cloud provider and need to set up the virtual networking

VPNs and Subnet Routers for Hybrid and Legacy Networks

A mesh VPN or subnet router bridges an encrypted network to a whole subnet, so an agent runtime can reach devices that cannot run a client themselves: older databases, appliances, and services that do not speak HTTP. Tailscale subnet routers are one example.

• Best for: hybrid and multi site networks, and legacy protocols
• The tradeoff: network level access is broad. A traditional VPN can put the agent on the same network as everything else, so pair it with strict policies that allow only specific hosts and ports, and plan for route management and overlapping address ranges

Private vs Hosted MCP Servers: When to Self Host

Self host an MCP server when its tools touch internal databases, credentials, business logic, or regulated data. A hosted server is fine for low risk tools that wrap public SaaS APIs. With a hosted server, prompts, tool arguments, and results flow through someone else's infrastructure and fall under their retention and residency terms.

Self hosting keeps data inside your boundary but moves the security work to you: TLS, token validation, audience checks, least privilege, and network isolation. A private server is not automatically a safe one, because an unauthenticated internal MCP server is open to anything that gets inside the network. For a deeper look at the control and cost tradeoffs, see this comparison of self hosted vs managed integration platforms.

Which Option to Choose

• Agent platform in the cloud and private services in your cloud accounts: private cloud links
• On premises systems behind strict firewalls: an outbound only tunnel
• Many legacy devices, mixed protocols, or multi site networks: a VPN subnet router with tight access rules
• Sensitive or regulated tools: a self hosted private MCP server, whichever network path you choose
• Most real deployments combine them: a private cloud link to a gateway, for example, which forwards to a tunnel for the on premises ERP

Why a Private Network Alone Isn't Enough

A private network controls who can reach a service, not what an agent does once it gets there. Prompt injection, confused deputy attacks, SSRF, and token passthrough all happen inside the perimeter, and an agent with access is already inside it.

Threats That Survive the Perimeter

• Prompt injection: instructions hidden in documents, emails, tool descriptions, or API responses steer the agent into calls nobody intended. The agent holds legitimate access, so the network sees a normal request.
• Confused deputy attacks: an attacker uses an authorized agent or proxy to do something they cannot do directly. The MCP security guidance describes how a proxy with a static client ID, dynamic client registration, and a consent cookie can be tricked into handing an attacker authorization codes without the user's consent.
• SSRF: while working out how to authenticate, an MCP client fetches URLs supplied by the server. A malicious or misconfigured server can point it at internal addresses or the cloud metadata endpoint, so the client becomes a proxy past your firewall and can leak cloud credentials.
• Token passthrough: an MCP server that accepts tokens issued for other services, or forwards them downstream, breaks trust boundaries and muddies audit trails. The MCP specification forbids it.
• Lateral movement: if every internal service trusts every other, one compromised agent or one unauthenticated internal MCP server exposes everything next to it.
• Excessive agency: OWASP's 2026 Top 10 for LLM Applications ranks it third, up from sixth in 2025, and traces it to too much functionality, too many permissions, and too much autonomy.

Controls That Still Apply Inside the Network

• Authenticate every request: the MCP authorization specification treats authorization as optional, which is why internal servers so often ship without it. Turn it on, even on a private network.
• Bind tokens to the target service: clients send a resource parameter when requesting tokens (RFC 8707), and servers must reject tokens that were not issued for them. A token for the HR service should be useless at the billing service.
• Filter egress: require HTTPS, block fetches to private, loopback, and link local ranges including the cloud metadata address, route outbound requests through an egress proxy with a destination allowlist, and watch for DNS rebinding. The MCP guidance also warns against writing IP validation by hand, because encoding tricks fool custom parsers.
• Allowlist tools: expose only what each agent needs, and keep destructive tools out of its catalog entirely.
• Use step up scopes: start with a minimal read scope, and when a privileged action is requested the server answers with a 403 that names the missing scope, so the client can ask for just that.

Authentication and Least Privilege for Internal APIs

Use short lived tokens, bound to the target service, that carry the user's identity when a person triggered the action, and a dedicated, narrowly scoped service identity when the agent acts alone. Avoid shared static keys wherever you can.

Choosing How Agents Authenticate

• Static API keys: the simplest to set up, but they act as a shared superuser, are hard to rotate, and leak into logs and context. Acceptable for a prototype, not for production access to internal systems.
• OAuth client credentials: the agent service authenticates as itself and receives a short lived token the API can validate for audience. Good for machine to machine calls, but the internal API sees the agent, not the person behind the request.
• Mutual TLS and workload identity (such as SPIFFE): each side proves its identity with certificates that rotate automatically, so you stop relying on IP addresses. It identifies the workload, not the user, and needs a certificate authority or service mesh.
• Token exchange (RFC 8693): swap the user's incoming token for a short lived token aimed at the target API that names both the user and the acting agent. It needs an authorization server that supports it. The MCP specification covers OAuth between client and server; token exchange is a separate standard teams add when they need delegation.

These are not mutually exclusive. A common pairing is mutual TLS to prove which workload is calling, plus an exchanged token to carry the user.

Acting for a User vs Acting as the Agent

• Delegation: the token carries the user as the subject (sub) and the agent as the actor (act). Internal APIs apply the user's own permissions, and audit logs show both parties.
• Impersonation: the token carries the user but not the actor, so the agent becomes indistinguishable from the person. That makes audits and incident response harder, so avoid it.
• The agent's own identity: use this for scheduled and background work where no user is present. Give it a dedicated service account with a narrow role, ideally read only.

This matters most with chains of agents. Indicium's chief data officer told CIO that every request is traced back to the person who asked, and that preserving that identity through the whole chain is hard, especially with legacy systems.

Scoping Access by Agent, Tool, Resource, and Record

• Agent: give each agent its own identity so you can grant, audit, and revoke it individually.
• Tool: allowlist the tools each agent may call, and keep read tools separate from write tools.
• Resource: bind each token to one service through its audience, with separate scopes per service.
• Record: tool code takes the tenant and user from the verified session, never from model arguments, and queries run through row level security or permission checks, so the agent only sees what the user is allowed to see.

What Should Be Read Only, Write Enabled, or Need Human Approval

• Read only, autonomous: tool discovery, status lookups, parameterized queries, and documentation search. Use a minimal read scope and row level checks.
• Write enabled, bounded: low risk, reversible, validated writes such as drafting a record, updating staging data, or writing an audit entry. Require strict schema validation, least privilege backend roles, and idempotency keys.
• Human approval: payments, refunds, deleting records, changing access or security settings, production configuration changes, and anything you cannot roll back. Execution pauses before the request is sent, and a reviewer sees the tool name and the proposed arguments.

You can build this yourself or use tooling that has it built in. Corsair, for example, lets you gate destructive actions behind human approval before they execute.

Connecting Agents to Legacy Systems and Enterprise Data

Put an anti corruption layer between the agent and the legacy system: a thin wrapper that translates model friendly tool calls into whatever the old system understands, validates what goes in, shapes what comes out, and decides what the agent may read or write. The agent never sees the raw interface.

Legacy systems were built for deterministic callers, and agents are not deterministic. EY's Rakesh Malhotra told CIO that most systems of record are still legacy and often on premises, and that batch processing clashes with how quickly users expect agents to respond. The same CIO report cites a late 2024 Langbase survey of 3,400 developers in which 70% had problems integrating agents with existing systems, and a Tray.ai survey in which 42% of enterprises said they need access to eight or more data sources and 38% named integration complexity as the biggest barrier to scaling agents.

The Anti Corruption Layer as Your Tool Boundary

• Translate: turn the model's structured tool calls into SOAP requests, mainframe transactions, or stored procedure parameters.
• Shape data: prune a record with dozens of obscurely named fields down to the few the model needs, which saves context window and reduces confusion.
• Validate: check every argument against a strict schema before anything is sent.
• Separate read from write: reads run autonomously under minimal privilege, while writes sit behind separate, tightly controlled tools.
• Close over identity: take the user from the session, not from the prompt.

Over time a strangler fig approach lets you replace the legacy backend behind the wrapper without the agent noticing.

Handling Non Idempotent and Irreversible Operations

Assume the agent will retry, and design so a retry cannot apply the same change twice.

• Idempotency keys: derive a stable key from validated parameters so repeated calls return the original result.
• State checks: if the system has no idempotency support, read the current state before writing.
• Classify errors: retry transient failures such as timeouts, and fail fast on validation errors so the agent does not loop.
• Cap the loop: limit the number of steps and halt on low confidence.
• Compensating actions: where you cannot roll back, define the undo, such as a reversing entry or a cancellation, and run it if a multi step operation fails partway.
• Approval: irreversible actions require human approval, and the most dangerous tools should not exist in the agent's catalog at all.

Narrow Internal APIs vs Direct Database Access

Give the agent a narrow, single purpose API, not a database connection. Direct access exposes every table, column, and row to prompt injection and hallucinated queries, while a narrow tool exposes one typed question with validated parameters. It is the same reasoning behind D&B's rule against agents writing their own SQL.

• Wrap each query in a typed function and use parameterized queries
• Enforce row level security so results match the user's entitlements
• Take tenant and user identity from the session, never from the prompt

Minimizing What Tool Responses Return to the Model

• Prune fields: strip internal flags, system metadata, and personal data the question does not need.
• Check permissions at query time: for search and retrieval tools, confirm that the user may read each document before it reaches the model.
• Never return secrets: no keys, tokens, or connection strings in tool results or error messages.
• Mask sensitive values in logs: arguments and outputs often contain the same data you worked to protect.

Going to Production: Gateways, Audit Logging, and a Readiness Checklist

An MCP gateway governs the calls an agent makes to your internal tools: who is calling, which tool, with which arguments, and what happens next. An AI gateway governs the conversation with the model: prompts, model routing, token budgets, and content filters. You may want both, but only the first protects your internal APIs.

MCP Gateway vs AI Gateway: What Each One Governs

• AI gateway: sits between your application and model providers, handling prompt logging, PII redaction, token accounting, and per model rate limits. As Twingate points out, it never sees the agent's follow up call to an internal API.
• MCP gateway: sits between the agent runtime and your tools. It authenticates the caller, applies tool level policy, routes to the right server, holds credentials, logs every call, and can pause for approval. Our MCP gateway guide walks through the request path in detail.
• When you need one: a single agent calling a couple of trusted tools can connect directly. Once several agents, teams, or tenants share tools, or the tools touch sensitive data, a gateway saves you from rebuilding the same controls in every agent.

What to Log for Every Tool Call

• Identity: the user and the agent, plus session and trace IDs that tie the model's reasoning to downstream calls
• Tool details: the server, destination, method, tool name, arguments with sensitive values masked, and the result or error code
• Authorization events: the policy decision (allowed or denied) and, for gated actions, the approval request, reviewer, timestamp, and decision
• Network evidence: the source and destination of the request, confirming traffic stayed on the private path
• Alerts: rate limit spikes, repeated denials, and unusual tool usage

Readiness Checklist Before Production Access

Copy this list into your launch review:

• [ ] The model holds no credentials, connection strings, or network access.
• [ ] User identity flows through delegated, short lived tokens, with the agent named as the actor.
• [ ] Every internal API and MCP server validates the token audience.
• [ ] All tool traffic passes through a gateway or policy enforcement point.
• [ ] Each agent has a minimal tool allowlist and least privilege scopes, with step up for anything more.
• [ ] Write tools that move money, change records, or alter access require human approval.
• [ ] Tool code validates every argument against a strict schema and treats model output as untrusted.
• [ ] Mutating calls use idempotency keys or state checks, and errors are classified as retryable or fatal.
• [ ] Internal systems are reachable only through private paths, with egress filtering on outbound fetches.
• [ ] Every call is logged with identity, arguments, decision, and outcome, with alerts on anomalies.

Conclusion: The Model Proposes, Trusted Code Executes

Connecting agents to internal systems is mostly a question of boundaries. Keep the model outside the trust boundary, reach private systems over paths that stay private, authenticate and authorize every call as if the network offered no protection, and put an anti corruption layer in front of anything old or irreversible. For the wider tour of tools, MCP, and integration approaches, the general guide to connecting agents to databases and APIs is the next stop.

Corsair is an open source integration layer for AI agents that you run inside your own app, so credentials stay encrypted in your own database instead of sitting with a third party. It exposes typed tools to agents over MCP, holds destructive actions for human approval before they execute, and gives each user or tenant their own isolated credentials. When your internal API has no connector, its plugin generator scaffolds one, so the tool your agent calls stays typed, permissioned, and running in your own infrastructure. See how it works at Corsair.

Frequently Asked Questions

Can AI agents access internal APIs without exposing them to the public internet? Yes. Keep the API private and give the agent a path to it: an outbound only tunnel from inside your network, a private cloud link such as AWS PrivateLink or Google Private Service Connect, or a VPN subnet router. The agent calls a private MCP server or gateway, and trusted tool code makes the request to the internal API, so nothing is published to the internet.

Should an AI agent use an API key or OAuth to call internal systems?
Prefer OAuth. Short lived tokens bound to the target service limit the damage if one leaks, and token exchange can carry the user's identity so internal systems apply that person's permissions. Static API keys are acceptable for prototypes, but they act as a shared superuser, are hard to rotate, and are more likely to leak into logs or prompts.

Do I need a gateway to connect agents to private systems?
Not for a single agent calling a couple of trusted tools. Once several agents, teams, or tenants share tools, or the tools touch sensitive data, a gateway centralizes authentication, tool level permissions, approvals, and audit logs so you do not rebuild them in every agent.

How do I connect an AI agent to a legacy ERP or SOAP service?
Wrap it in an anti corruption layer: a small service that exposes a few typed tools to the agent and translates them into SOAP calls or stored procedures. Validate every argument, return only the fields the model needs, keep read tools separate from write tools, use idempotency keys for updates, and require human approval for anything irreversible.

Is a VPN enough to secure an AI agent's access to internal systems?
No. A VPN limits who can reach the network, but not what an agent does with that access. Prompt injection, confused deputy attacks, SSRF, and overly broad tokens all work from inside the perimeter. Add per request authentication, audience bound tokens, tool allowlists, egress filtering, and audit logging.