> ## Documentation Index
> Fetch the complete documentation index at: https://docs.openlayer.com/llms.txt
> Use this file to discover all available pages before exploring further.
# A2A Protocol
> Trace multi-agent systems that communicate over the A2A protocol — as one merged trace, or as linked per-project traces
The [Agent2Agent (A2A) protocol](https://a2a-protocol.org/) is an open standard for
agent-to-agent communication: one agent (the client) sends tasks to another agent (the
server) over HTTP, even when the two agents are built by different teams and deployed as
separate services.
Because the A2A SDKs ride on standard HTTP, OpenTelemetry trace context propagates
across the A2A hop automatically: the calling agent injects the W3C `traceparent` header,
the serving agent picks it up, and the spans from **both processes share one trace ID**.
This works with both the [A2A Python SDK](https://a2a-protocol.org/latest/sdk/python/)
(which additionally ships its own OTel telemetry) and the
[A2A JavaScript SDK](https://github.com/a2aproject/a2a-js), where propagation comes from
the standard OTel HTTP instrumentations. Export both agents' telemetry to Openlayer and
you get a single trace that starts at the user request in the calling agent and continues
through the serving agent's LLM and tool calls — the full distributed picture of your
multi-agent system.
Agents that export to the **same inference pipeline** (see the `x-bt-parent`
header below) merge into one trace — that is the recipe on this page, and it
works with no further setup. To give each agent its **own project** and
navigate between the linked traces, see [One project per
agent](#one-project-per-agent-linked-traces): the instrumentation is
identical, only the export destination changes.
## Installation
```bash Python theme={null}
pip install "a2a-sdk[http-server]" \
opentelemetry-sdk \
opentelemetry-exporter-otlp-proto-http \
opentelemetry-instrumentation-httpx \
opentelemetry-instrumentation-starlette \
opentelemetry-processor-baggage
```
```bash TypeScript theme={null}
npm install @a2a-js/sdk express \
@opentelemetry/api \
@opentelemetry/sdk-node \
@opentelemetry/resources \
@opentelemetry/sdk-trace \
@opentelemetry/exporter-trace-otlp-proto \
@opentelemetry/baggage-span-processor \
@opentelemetry/instrumentation-http \
@opentelemetry/instrumentation-undici
```
## 1. Configure the OpenTelemetry exporter — in every agent
Each agent runs the standard OTel SDK and exports to
[Openlayer's OTel endpoint](/integrations/opentelemetry#opentelemetry-endpoint). The
snippet below also installs a `BaggageSpanProcessor`, which copies `openlayer.*`
[baggage](https://opentelemetry.io/docs/concepts/signals/baggage/) entries (session and
user IDs — see [below](#propagate-session-and-user-context-with-baggage)) onto every span
the process creates.
```python Python (tracing.py) theme={null}
import os
from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.processor.baggage import BaggageSpanProcessor
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
def init_openlayer_tracing(service_name: str) -> TracerProvider:
provider = TracerProvider(resource=Resource.create({"service.name": service_name}))
# Stamp openlayer.* baggage (session/user IDs) onto every local span
provider.add_span_processor(
BaggageSpanProcessor(lambda key: key.startswith("openlayer."))
)
provider.add_span_processor(
BatchSpanProcessor(
OTLPSpanExporter(
endpoint="https://api.openlayer.com/v1/otel/v1/traces",
headers={
"Authorization": f"Bearer {os.environ['OPENLAYER_API_KEY']}",
"x-bt-parent": f"pipeline_id:{os.environ['OPENLAYER_INFERENCE_PIPELINE_ID']}",
},
)
)
)
trace.set_tracer_provider(provider)
return provider
```
```typescript TypeScript (instrumentation.ts) theme={null}
import { NodeSDK } from "@opentelemetry/sdk-node";
import { resourceFromAttributes } from "@opentelemetry/resources";
import { BatchSpanProcessor } from "@opentelemetry/sdk-trace";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-proto";
import { BaggageSpanProcessor } from "@opentelemetry/baggage-span-processor";
import { HttpInstrumentation } from "@opentelemetry/instrumentation-http";
import { UndiciInstrumentation } from "@opentelemetry/instrumentation-undici";
const sdk = new NodeSDK({
resource: resourceFromAttributes({ "service.name": process.env.SERVICE_NAME }),
spanProcessors: [
// Stamp openlayer.* baggage (session/user IDs) onto every local span
new BaggageSpanProcessor((key) => key.startsWith("openlayer.")),
new BatchSpanProcessor({
exporter: new OTLPTraceExporter({
url: "https://api.openlayer.com/v1/otel/v1/traces",
headers: {
Authorization: `Bearer ${process.env.OPENLAYER_API_KEY}`,
"x-bt-parent": `pipeline_id:${process.env.OPENLAYER_INFERENCE_PIPELINE_ID}`,
},
}),
}),
],
instrumentations: [
new HttpInstrumentation({
// Keep health checks and agent-card fetches out of your records
ignoreIncomingRequestHook: (req) =>
req.url?.startsWith("/.well-known") || req.url === "/healthz",
}),
new UndiciInstrumentation(),
],
});
sdk.start();
```
In TypeScript, load the instrumentation before your application code so the
HTTP patches apply: `node --import ./instrumentation.js server.js`. The
`UndiciInstrumentation` covers the global `fetch` the A2A client uses; the
`HttpInstrumentation` covers the Express server.
## 2. Instrument the calling agent (A2A client)
The A2A client rides on the language's standard HTTP stack — `httpx` in Python, `fetch`
(undici) in Node — so instrumenting that stack is what injects the `traceparent` and
`baggage` headers into the A2A request:
```python Python theme={null}
from opentelemetry import baggage, context, trace
from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor
import a2a.types as a2a_types
from a2a.client import ClientConfig, create_client, minimal_agent_card
from a2a.helpers import get_stream_response_text, new_text_message
from tracing import init_openlayer_tracing # step 1
init_openlayer_tracing("orchestrator-agent")
HTTPXClientInstrumentor().instrument() # injects traceparent + baggage on the A2A hop
tracer = trace.get_tracer(__name__)
async def handle_user_request(question: str, session_id: str, user_id: str) -> str:
# Put the session/user IDs in baggage so BOTH agents' spans carry them
ctx = baggage.set_baggage("openlayer.session.id", session_id)
ctx = baggage.set_baggage("openlayer.user.id", user_id, context=ctx)
token = context.attach(ctx)
try:
# The root span of the distributed trace
with tracer.start_as_current_span("handle_user_request"):
client = await create_client(
minimal_agent_card("http://worker-agent:9999", transports=["JSONRPC"]),
ClientConfig(streaming=False),
)
request = a2a_types.SendMessageRequest(
message=new_text_message(question, role=a2a_types.Role.ROLE_USER)
)
parts = [
get_stream_response_text(response)
async for response in client.send_message(request)
]
await client.close()
return " ".join(filter(None, parts))
finally:
context.detach(token)
```
```typescript TypeScript theme={null}
import { randomUUID } from "node:crypto";
import { context, propagation, trace } from "@opentelemetry/api";
import { ClientFactory, JsonRpcTransportFactory } from "@a2a-js/sdk/client";
const tracer = trace.getTracer("orchestrator-agent");
export async function handleUserRequest(
question: string,
sessionId: string,
userId: string
): Promise {
// Put the session/user IDs in baggage so BOTH agents' spans carry them
const bag = propagation.createBaggage({
"openlayer.session.id": { value: sessionId },
"openlayer.user.id": { value: userId },
});
const ctx = propagation.setBaggage(context.active(), bag);
return context.with(ctx, () =>
// The root span of the distributed trace
tracer.startActiveSpan("handle_user_request", async (span) => {
const factory = new ClientFactory({
transports: [new JsonRpcTransportFactory()],
});
const client = await factory.createFromUrl("http://worker-agent:9999");
const result = await client.sendMessage({
tenant: "",
message: {
messageId: randomUUID(),
contextId: "",
taskId: "",
role: 1, // ROLE_USER
parts: [
{
content: { $case: "text", value: question },
metadata: undefined,
filename: "",
mediaType: "",
},
],
metadata: undefined,
extensions: [],
referenceTaskIds: [],
},
configuration: undefined,
metadata: undefined,
});
// SendMessageResult is Message | Task; a plain reply comes back as a Message
const text =
"parts" in result
? result.parts
.map((p) => (p.content?.$case === "text" ? p.content.value : ""))
.join(" ")
: "";
span.end();
return text;
})
);
}
```
## 3. Instrument the serving agent (A2A server)
On the server side, instrumenting the HTTP server (Starlette in Python, Express via the
Node `http` module in TypeScript) is what extracts the incoming `traceparent` and
`baggage` headers, so every span this agent creates — including LLM calls made inside
your `AgentExecutor` — joins the caller's trace:
```python Python theme={null}
from opentelemetry.instrumentation.starlette import StarletteInstrumentor
from starlette.applications import Starlette
from a2a.client import minimal_agent_card
from a2a.helpers import get_message_text, new_text_message
from a2a.server.agent_execution import AgentExecutor, RequestContext
from a2a.server.events import EventQueue
from a2a.server.request_handlers import DefaultRequestHandler
from a2a.server.routes import create_agent_card_routes, create_jsonrpc_routes
from a2a.server.tasks import InMemoryTaskStore
from tracing import init_openlayer_tracing # step 1
init_openlayer_tracing("worker-agent")
class WorkerAgentExecutor(AgentExecutor):
async def execute(self, context: RequestContext, event_queue: EventQueue) -> None:
question = get_message_text(context.message)
# Your agent logic. LLM calls made here (via any OTel-instrumented
# SDK or framework) show up as steps inside the distributed trace.
answer = await run_agent(question)
await event_queue.enqueue_event(
new_text_message(
answer, context_id=context.context_id, task_id=context.task_id
)
)
async def cancel(self, context: RequestContext, event_queue: EventQueue) -> None:
pass
card = minimal_agent_card("http://worker-agent:9999", transports=["JSONRPC"])
card.name = "worker-agent"
handler = DefaultRequestHandler(
agent_executor=WorkerAgentExecutor(),
task_store=InMemoryTaskStore(),
agent_card=card,
)
app = Starlette(
routes=create_jsonrpc_routes(handler, rpc_url="/") + create_agent_card_routes(card)
)
StarletteInstrumentor().instrument_app(app) # extracts traceparent + baggage
```
```typescript TypeScript theme={null}
import { randomUUID } from "node:crypto";
import express from "express";
import {
AgentEvent,
DefaultRequestHandler,
InMemoryTaskStore,
type AgentExecutor,
type ExecutionEventBus,
type RequestContext,
} from "@a2a-js/sdk/server";
import {
UserBuilder,
agentCardHandler,
jsonRpcHandler,
} from "@a2a-js/sdk/server/express";
const agentCard = {
name: "worker-agent",
description: "Worker agent",
version: "1.0.0",
supportedInterfaces: [
{
url: "http://worker-agent:9999/",
protocolBinding: "JSONRPC",
tenant: "",
protocolVersion: "1.0",
},
],
capabilities: { streaming: false, extensions: [] },
provider: undefined,
skills: [],
signatures: [],
securitySchemes: {},
securityRequirements: [],
defaultInputModes: ["text/plain"],
defaultOutputModes: ["text/plain"],
};
const executor: AgentExecutor = {
async execute(requestContext: RequestContext, eventBus: ExecutionEventBus) {
const question = (requestContext.userMessage?.parts ?? [])
.map((p) => (p.content?.$case === "text" ? p.content.value : ""))
.join(" ");
// Your agent logic. LLM calls made here (via any OTel-instrumented
// SDK or framework) show up as steps inside the distributed trace.
const answer = await runAgent(question);
eventBus.publish(
AgentEvent.message({
messageId: randomUUID(),
contextId: requestContext.contextId,
taskId: "",
role: 2, // ROLE_AGENT
parts: [
{
content: { $case: "text", value: answer },
metadata: undefined,
filename: "",
mediaType: "",
},
],
metadata: undefined,
extensions: [],
referenceTaskIds: [],
})
);
eventBus.finished();
},
async cancelTask() {},
};
const handler = new DefaultRequestHandler(
agentCard,
new InMemoryTaskStore(),
executor
);
const app = express();
app.use(
"/.well-known/agent-card.json",
agentCardHandler({ agentCardProvider: handler })
);
app.use(
jsonRpcHandler({ requestHandler: handler, userBuilder: UserBuilder.noAuthentication })
);
app.listen(9999);
```
Run the worker (any ASGI server such as `uvicorn` for Python; `node --import
./instrumentation.js server.js` for TypeScript). When the orchestrator handles a
user request, Openlayer shows **one record** whose trace tree starts at
`handle_user_request`, crosses the A2A hop, and continues through the worker agent's
execution — LLM steps, token counts, and latency included:
## One project per agent (linked traces)
Everything above merges the agents into one trace in one pipeline. When different teams
own the agents — or you want per-agent records, tests, and dashboards — point each
agent's exporter at **its own project's pipeline** instead. The instrumentation does not
change at all: only the `x-bt-parent` header differs per agent.
With trace context propagating exactly as before, Openlayer links the per-project traces:
1. The serving agent's spans arrive with a parent that lives in another process (the
caller). Openlayer **promotes** that subtree into a full record in the serving agent's
own project — with its own step tree, tokens, and cost — instead of leaving it
unreachable.
2. The severed parent edge is preserved, so the two records are linked **directionally**:
the caller's record shows a *Continues in …* chip and the serving agent's record shows
a *Called from …* chip. Each chip deep-links to the other record.
A few properties worth knowing:
* **Links are direct call edges only.** An orchestrator that fans out to several agents
shows one chip per agent it called; two agents that merely share a trace without
calling each other are not linked.
* **Works within one project too.** A single project with a dedicated pipeline per agent
behaves identically — chips are labeled with the peer pipeline's name.
* **Scoped to your workspace.** Link lookups only ever consider pipelines in the same
workspace, in projects the viewer can access.
Per-project linking relies on remote-root promotion, which is enabled per
workspace. Reach out to your Openlayer contact to turn it on; without it,
point all agents at one pipeline (the recipe above).
## Using Google ADK agents over A2A
[Google ADK](https://google.github.io/adk-docs/) agents speak A2A natively: `to_a2a()`
exposes an agent as an A2A server, and `RemoteA2aAgent` consumes one as a sub-agent. The
OTel setup from steps 1–3 applies unchanged, so an ADK orchestrator and an ADK worker in
separate projects produce linked traces with real `invoke_agent`, LLM, and tool steps on
both sides. (For tracing a single ADK app without A2A, the
[Google ADK integration](/integrations/google-adk) is the simpler path.)
```bash theme={null}
pip install "google-adk[a2a]" "a2a-sdk[http-server]"
```
Expose the worker agent — wrapping the ASGI app directly keeps per-event spans and
agent-card fetches out of your records:
```python worker.py theme={null}
from google.adk.a2a.utils.agent_to_a2a import to_a2a
from google.adk.agents import Agent
from opentelemetry.instrumentation.asgi import OpenTelemetryMiddleware
from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor
from tracing import init_openlayer_tracing # step 1
init_openlayer_tracing("currency-worker")
HTTPXClientInstrumentor().instrument() # outbound model calls
currency_worker = Agent(
model="gemini-flash-latest",
name="currency_worker",
description="Converts amounts between currencies.",
instruction="Use the get_exchange_rate tool, then answer concisely.",
tools=[get_exchange_rate],
)
app = OpenTelemetryMiddleware( # extracts traceparent + baggage
to_a2a(currency_worker, port=9999),
excluded_urls=".*/.well-known/.*,healthz",
exclude_spans=["receive", "send"],
)
```
Consume it from the orchestrator. ADK's `invoke_agent` span for a remote agent carries no
content of its own (the request and response travel inside the A2A message body), so the
optional callback below stamps them onto the span with standard attributes — the handoff
step then shows exactly what was asked of the remote agent and what it answered:
```python orchestrator.py theme={null}
import json
from google.adk.agents import Agent
from google.adk.agents.remote_a2a_agent import RemoteA2aAgent
from opentelemetry import trace
from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor
from tracing import init_openlayer_tracing # step 1
init_openlayer_tracing("travel-orchestrator")
HTTPXClientInstrumentor().instrument() # A2A hop + model calls
def _text_of(content) -> str:
if not content or not content.parts:
return ""
return "\n".join(part.text for part in content.parts if part.text)
def stamp_handoff_io(callback_context):
"""Record the remote agent's request/response on its handoff step."""
span = trace.get_current_span()
if not span.is_recording():
return None
question = _text_of(callback_context.user_content)
if question:
span.set_attribute(
"gen_ai.input.messages",
json.dumps([{"role": "user", "parts": [{"type": "text", "content": question}]}]),
)
for event in reversed(callback_context.session.events):
if event.author == callback_context.agent_name:
answer = _text_of(event.content)
if answer:
span.set_attribute(
"gen_ai.output.messages",
json.dumps(
[{"role": "assistant", "parts": [{"type": "text", "content": answer}]}]
),
)
break
return None
currency_worker = RemoteA2aAgent(
name="currency_worker",
description="Converts amounts between currencies.",
agent_card="http://127.0.0.1:9999/.well-known/agent-card.json",
after_agent_callback=stamp_handoff_io,
)
orchestrator = Agent(
model="gemini-flash-latest",
name="travel_orchestrator",
instruction="For currency questions, delegate to currency_worker.",
sub_agents=[currency_worker],
)
```
ADK-specific gotchas:
* **The agent-card URL must match the worker's serving origin exactly** — resolving a
card served on `localhost` via `127.0.0.1` (or vice versa) fails with a card
resolution error.
* **Set `OTEL_PYTHON_EXCLUDED_URLS` on the worker** (the middleware above does it in
code): otherwise the orchestrator's agent-card fetch becomes a record of its own in
the worker's project.
* `google-adk[a2a]` does not pull the A2A server extra — install
`a2a-sdk[http-server]` alongside it, as shown above.
## Propagate session and user context with Baggage
Openlayer reads the `openlayer.session.id` and `openlayer.user.id` span attributes to
group traces into [sessions and users](/monitoring/sessions-and-users). In a multi-agent
system, only the entry-point agent knows those IDs — W3C Baggage is how they reach the
other agents:
1. The calling agent puts the IDs in baggage (step 2 above).
2. The instrumented HTTP client sends them in the `baggage` header alongside
`traceparent`.
3. Each agent's `BaggageSpanProcessor` (step 1 above) copies them onto every span it
creates.
The result: every span from every agent carries the session and user, with no
Openlayer-specific code in the downstream agents.
Baggage travels in plaintext HTTP headers to **every** downstream service the
instrumented client calls — including third-party APIs. Filter which keys you
propagate (the snippet above only copies `openlayer.*`) and keep sensitive
values out of baggage.
## Tips
* **Exclude health checks and agent-card fetches.** Server-side HTTP instrumentation
turns every direct request — including `/.well-known/agent-card.json` polls and health
checks — into a root span, which becomes a record in Openlayer. In Python, exclude
those URLs via the environment (the TypeScript setup above already handles this with
`ignoreIncomingRequestHook`):
```bash theme={null}
OTEL_PYTHON_EXCLUDED_URLS=".*/.well-known/.*,healthz"
```
* **Trim the A2A Python SDK's internal spans.** The Python SDK instruments its own
internals (event queues, dispatchers), which adds a couple dozen spans per message. If
you prefer leaner traces, disable them and rely on the `httpx`/Starlette
instrumentation for propagation (the JavaScript SDK emits no internal spans, so its
traces are lean by default):
```bash theme={null}
OTEL_INSTRUMENTATION_A2A_SDK_ENABLED=false
```
* **Configuring the exporter via environment variables instead?** Use
`OTEL_EXPORTER_OTLP_ENDPOINT=https://api.openlayer.com/v1/otel` (the SDK appends
`/v1/traces`), as shown on the [OpenTelemetry integration
page](/integrations/opentelemetry#opentelemetry-endpoint).