AI Mesh: What Changes When Agents Need to Work Together?

articleOctober 10, 2026Updated October 28, 2026By Ali Haghighi

Building Enterprise AI Systems, part 1

One incident agent is manageable. Three agents owned by different teams are a different engineering problem.

Imagine one team building an agent that investigates production incidents. Another team builds one that understands deployments. A third has an agent for searching internal documentation. Each works well on its own.

Then someone asks the incident agent a reasonable question: Did yesterday's deployment cause the latency increase, and have we seen this failure before?

Now the incident agent needs information owned by other teams. We can add a few HTTP calls, share some tools, and get an answer. That might be all we need.

But suppose these integrations multiply. Agents move between environments, teams deploy independently, access policies differ, and no one can easily explain which agent called which service during an investigation.

At that point, improving the prompt won't solve the integration problem. We need to decide how separately owned capabilities find each other, authenticate, and fail without bringing the whole workflow down.

This is where the idea of an AI Mesh, often called an Agent Mesh, becomes relevant.

What do we actually mean by AI Mesh?

I think of AI Mesh as an architectural approach for connecting agents, tools, and sometimes models through shared infrastructure, rather than building a new point-to-point integration for every interaction.

Depending on the implementation, that infrastructure might handle:

Discovery: Which capabilities are available, and where?
Identity and access: Which agent is making a request, and what is it allowed to do?
Connectivity and routing: How do agents reach one another across networks and environments?
Observability: Can we follow a request across multiple agents and services?
Governance: Can teams apply consistent policies without owning every agent?

That description is deliberately broad. AI Mesh is not one agreed-upon protocol or a universally standardized architecture. Different projects use the term for different combinations of networking, agent discovery, policy enforcement, orchestration, and model access.

A concrete example is Google's open-source Agent Mesh project, which focuses on a private network where nodes can publish and discover capabilities, verify identity, and enforce access policies. It is an implementation of a particular vision, not a definition every AI Mesh must follow.

The distinction matters. A mesh should solve real coordination and operational problems, not become a new name for a collection of API endpoints.

Why not just connect agents directly?

For a small system, direct integration is usually simpler.

An incident agent can call an MCP server for metrics, query a deployment API, and retrieve documentation. Everything stays in one workflow, with a manageable number of dependencies.

The difficulty starts when the capabilities become independently owned and reused.

Consider three teams, each with its own agent. Every pair may need to agree on endpoints, credentials, deployment details, and failure handling. Add more teams and environments, and coordination becomes increasingly expensive.

There is also an ownership issue. If the documentation team changes how its agent runs, should every consumer need to update its integration? If one agent needs access to production monitoring but another should only see a test environment, where is that policy enforced?

A mesh can make some of these concerns shared infrastructure instead of repeated application code.

It does not remove the need for good contracts, authorization decisions, or operational discipline. It creates a place to manage them more consistently.

MCP, A2A, AI Gateway, and Mesh are not the same thing

These terms are sometimes discussed as if they were alternatives. They address different layers of the problem.

| Concept | The question it primarily answers | | --- | --- | | MCP (Model Context Protocol) | How does an AI application access tools and contextual data through a standard interface? | | A2A (Agent2Agent) | How can independent agents discover capabilities, exchange messages, and collaborate on tasks? | | AI Gateway | How do applications access AI models under common routing, access, usage, and cost controls? | | AI Mesh / Agent Mesh | How do we connect, secure, discover, and operate distributed AI capabilities across teams and environments? |

A useful mental model is that MCP connects agents to tools, while A2A connects independent agents to each other. The A2A specification explicitly treats these as complementary approaches.

An AI gateway is generally concerned with model-facing traffic: routing requests, enforcing limits, and measuring usage. Exact features depend on the product.

A mesh works at a broader infrastructure level. It might use MCP or A2A without replacing them. It might also integrate with an AI gateway when model access needs to be centrally managed.

There is overlap. For example, A2A already defines mechanisms for agent capability discovery. So it would be misleading to say that discovery is something only a mesh provides. The mesh question is how discovery, identity, connectivity, and policies work consistently across a larger environment.

A simple architecture sketch

Think of a production-support application that needs three capabilities:

TEXT
                 Incident Investigation
                          |
                 Workflow / Coordinator
                          |
              Shared connectivity & policy
                   (mesh, if needed)
                   /       |       \
             Metrics   Deployments  Knowledge
              Agent       Agent       Agent
                |           |           |
             Monitoring   CI/CD      Runbooks

The coordinator still owns the investigation. It decides what evidence is needed, which capability to invoke, and how to present the result.

The mesh, where justified, handles cross-cutting infrastructure concerns such as who can reach a capability and how that interaction is secured and traced.

This is an important boundary: the mesh is not the reasoning engine. Adding one does not automatically make agents better at diagnosing incidents, selecting tools, or evaluating evidence.

Nor does every box in this diagram need to be an agent. A metrics query may be better exposed as a plain MCP tool. An independently operated deployment-analysis service might justify an agent interface. The right boundary depends on responsibility and ownership, not on making everything agentic.

The connection to service mesh

The name naturally invites comparison with service mesh.

Traditional service meshes help manage communication between distributed services, commonly addressing traffic policy, identity, security, and telemetry.

AI Mesh borrows some of that thinking, but AI workloads have additional concerns. An agent may perform several steps for a single request, call models repeatedly, delegate tasks, and return an answer that is syntactically valid but unsupported by evidence.

Those behaviors introduce problems that network infrastructure alone cannot solve.

A trace might tell us that Agent A called Agent B successfully. It does not tell us whether Agent B's conclusion was correct.

That requires application-level evaluation, evidence provenance, and well-defined acceptance criteria. Infrastructure visibility and answer quality are two different things.

When does a mesh actually make sense?

I would start considering it when several of these are true:

Different teams deploy and own their agents independently.
Capabilities are shared by multiple applications.
Agents communicate across security or environment boundaries.
Point-to-point integrations are becoming hard to maintain.
Identity, authorization, and tracing need consistent enforcement.

If none of these conditions applies, I would probably avoid it.

A single agent calling a handful of well-designed tools is often easier to test and operate than a distributed network of agents. Even when a future mesh seems likely, stable interfaces and clear ownership boundaries are more useful early investments than adopting another infrastructure layer immediately.

There is also a cost to centralizing infrastructure: new operational dependencies, policy configuration, troubleshooting paths, and potentially new failure modes. The mesh needs to reduce more complexity than it introduces.

Where I land

I see AI Mesh less as a new way to make agents intelligent and more as an attempt to make networks of independently operated AI capabilities manageable.

That is a meaningful problem. It is also a problem many teams do not have yet.

My default would be to start with one well-scoped application, expose tools through clean contracts, and make the workflow observable. As teams begin sharing independently deployed agents, introduce interoperability where it is needed. Evaluate mesh infrastructure only when connectivity, identity, discovery, and governance are becoming repeated operational problems.

The architecture should grow because the boundaries are real, not because a diagram looks more advanced with another layer.

In Part 2, the incident example becomes a design exercise: which capabilities should remain tools, which justify independent agents, and what does a distributed boundary actually cost?