> ## Documentation Index
> Fetch the complete documentation index at: https://docs.openlayer.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Google Agent Development Kit (ADK)

> Learn how to evaluate multi-agent systems built with Google Agent Development Kit using Openlayer

If you are building AI systems with [Google Agent Development Kit (ADK)](https://developers.google.com/adk) and want to evaluate multi-agent conversations, handoffs, and tool usage, you can use the [SDKs](/api-reference/sdk/overview) to make Openlayer part of your workflow.

This integration guide shows how you can comprehensively trace and monitor your multi-agent systems powered by Gemini models.

## Choosing an integration path

Openlayer supports **two** ways to instrument Google ADK. Both land traces in the same Openlayer pipeline; pick the one that fits your stack.

| Path                                  | Setup                                                                         | When to use it                                                                                                                          |
| ------------------------------------- | ----------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| **1. Openlayer tracer** (recommended) | One call — `init()`                                                           | You want the richest Openlayer-native metadata (agent instructions, tool I/O, session context) and the least setup.                     |
| **2. ADK OpenTelemetry**              | Point ADK's built-in OTel exporter at Openlayer's OTLP endpoint (ADK 1.17.0+) | You already standardize on OpenTelemetry — for example an existing collector, or a policy that forbids vendor SDKs in application code. |

The rest of this page walks through each path. **If you're not sure, start with the Openlayer tracer.**

Do not enable both paths in the same process. See [Don't combine both paths](#dont-combine-both-paths).

## Path 1 — Openlayer tracer (recommended)

A single `init()` call auto-instruments Google ADK (it runs `trace_google_adk()` under the hood) along with any other installed LLM SDKs.

### Monitoring

To use the [monitoring mode](/monitoring/overview), you must instrument your code to publish the requests your AI system receives to the Openlayer platform.

To set it up, you must follow the steps in the code snippet below:

```python Python theme={null}
# 1. Set the environment variables
import os

os.environ["OPENLAYER_API_KEY"] = "YOUR_OPENLAYER_API_KEY_HERE"
os.environ["OPENLAYER_INFERENCE_PIPELINE_ID"] = "YOUR_OPENLAYER_INFERENCE_PIPELINE_ID_HERE"

# 2. Call `init` to auto-instrument the installed LLM SDKs BEFORE creating agents
from openlayer.lib import init

init()

# 3. Create your agents with tools and sub-agents
from google.adk.agents import Agent
from google.adk.runners import Runner

def get_weather(location: str) -> str:
    """Get weather for a location."""
    return f"Sunny and 72°F in {location}"

weather_agent = Agent(
    name="weather_agent",
    model="gemini-2.0-flash-exp",
    description="Provides weather information",
    instructions="You help users get weather information for any location.",
    tools=[get_weather]
)

main_agent = Agent(
    name="main_agent",
    model="gemini-2.0-flash-exp",
    instructions="You are a helpful assistant that can check the weather.",
    sub_agents=[weather_agent]
)

# 4. Run conversations with automatic tracing
async def run_conversation(user_input: str):
    runner = Runner(main_agent)

    async for event in runner.run_async(
        session_id="session-123",
        user_id="user-456",
        message=user_input
    ):
        if event.is_final_response():
            print(event.content)

# From now on, all agent conversations, handoffs, and tool calls
# are automatically traced by Openlayer
await run_conversation("What's the weather in San Francisco?")
```

<Card title="See full Python example" icon="python" iconType="duotone" href="https://colab.research.google.com/github/openlayer-ai/openlayer-python/blob/main/examples/tracing/google-adk/google_adk_tracing.ipynb" />

Once the code is instrumented, all your Google ADK interactions are automatically published to Openlayer, including:

* **Agent execution** with agent names, descriptions, and instructions
* **LLM calls** to Gemini models with messages and configurations
* **Token usage** including prompt tokens, completion tokens, and totals
* **Tool calls** with function names, arguments, and results
* **Agent transfers** and handoffs between sub-agents with proper hierarchy
* **Session context** including user IDs, session IDs, and invocation tracking
* **Metadata** such as latency and timestamps for all operations

If you navigate to the "Data" page of your Openlayer data source, you can see the complete traces for each multi-agent conversation.

<Note>
  The Google ADK integration automatically captures the full agent workflow,
  including sub-agent handoffs and tool usage. You can use this together with
  [tracing](/monitoring/tracing) to monitor complex multi-agent systems as part
  of larger AI workflows. Make sure to call `init()` **before** creating any
  agents.
</Note>

After your AI system requests are continuously published and logged by Openlayer, you can [create tests](/tests/overview) that run at a regular cadence on top of them.

Refer to the [Monitoring overview](/monitoring/overview), for details on Openlayer's monitoring mode, to the [Publishing data guide](/monitoring/publishing-data), for more information on setting it up, or to the [Tracing guide](/monitoring/tracing), to understand how to trace more complex systems.

## Path 2 — ADK OpenTelemetry

As of ADK 1.17.0, the framework ships its own OpenTelemetry instrumentation ([ADK traces](https://github.com/google/adk-docs/blob/main/docs/observability/traces.md), [Cloud Observability](https://docs.cloud.google.com/stackdriver/docs/instrumentation/ai-agent-adk)) and can export traces over OTLP to any backend — including Openlayer. This is the path to use when you already run an OTel collector, or when application code is not allowed to depend on a vendor tracing SDK.

Openlayer ingests the spans via its [OpenTelemetry endpoint](/integrations/opentelemetry). ADK emits [GenAI semantic-convention](https://opentelemetry.io/docs/specs/semconv/gen-ai/) spans such as `invoke_agent`, `execute_tool`, and `generate_content`, which Openlayer maps onto agent, tool, and chat-completion steps.

### Install packages

```bash theme={null}
pip install "google-adk>=1.17.0" \
            opentelemetry-instrumentation-google-genai \
            opentelemetry-instrumentation-vertexai \
            opentelemetry-exporter-otlp-proto-http
```

<Note>
  Openlayer's OTel endpoint accepts **OTLP HTTP/protobuf** (not gRPC). Install
  `opentelemetry-exporter-otlp-proto-http`, not the gRPC exporter Google's Cloud
  Trace samples use. See the [OpenTelemetry integration
  page](/integrations/opentelemetry) for the full endpoint reference.
</Note>

The Google GenAI and Vertex AI instrumentors enrich the underlying model calls (tokens, model name, messages). ADK itself emits the agent and tool spans.

### Configure the environment

Set the following environment variables **before** constructing any ADK agents. You can put them in the process environment, a dotenv file, or `os.environ` at process start.

```bash theme={null}
# OTLP/HTTP -> Openlayer (traces only)
OTEL_EXPORTER_OTLP_PROTOCOL="http/protobuf"
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT="https://api.openlayer.com/v1/otel/v1/traces"
OTEL_EXPORTER_OTLP_TRACES_HEADERS="Authorization=Bearer YOUR_OPENLAYER_API_KEY, x-bt-parent=pipeline_id:YOUR_OPENLAYER_PIPELINE_ID"
OTEL_TRACES_EXPORTER="otlp"
OTEL_METRICS_EXPORTER="none"
OTEL_LOGS_EXPORTER="none"

# Latest GenAI semantic conventions (required for SPAN_ONLY / EVENT_ONLY)
OTEL_SEMCONV_STABILITY_OPT_IN="gen_ai_latest_experimental"

# Put prompts and responses on span attributes so Openlayer can read them
OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT="SPAN_ONLY"

# Required: put message content on ADK's own spans. Without this, Input /
# Output are empty and Openlayer cannot evaluate the traces.
ADK_CAPTURE_MESSAGE_CONTENT_IN_SPANS="true"

# Optional: resource label only — does not select the Openlayer pipeline
OTEL_SERVICE_NAME="adk-agent"
```

Replace `YOUR_OPENLAYER_API_KEY` and `YOUR_OPENLAYER_PIPELINE_ID` with values from your Openlayer workspace. The `x-bt-parent` header is what routes traces to the correct inference pipeline — not `OTEL_SERVICE_NAME`.

<Warning>
  **Do not set `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true` when
  using the latest GenAI semantic conventions.** `true` is not a valid value for
  that mode. The GenAI instrumentor silently captures **no** prompt or response
  content — traces still arrive, but Input / Output in Openlayer are empty. Use
  `SPAN_ONLY` (for Openlayer) or `EVENT_ONLY` (for log-oriented backends such as
  Cloud Logging).
</Warning>

<Note>
  Google's Cloud Observability docs recommend
  `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=EVENT_ONLY` and
  `ADK_CAPTURE_MESSAGE_CONTENT_IN_SPANS=false`, which keep prompts and responses
  off span attributes (they go to log records instead). Openlayer's OTel
  endpoint ingests **traces**, not logs, and tests need Input / Output on those
  traces. Use `SPAN_ONLY` **and** `ADK_CAPTURE_MESSAGE_CONTENT_IN_SPANS=true` so
  content lands on the spans Openlayer maps (`gen_ai.input.messages` /
  `gen_ai.output.messages` and ADK's own span attributes).
</Note>

Openlayer does not ingest OTLP logs or metrics, so `OTEL_LOGS_EXPORTER=none` and `OTEL_METRICS_EXPORTER=none` avoid failed export attempts. Google's `OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED` setting is for Cloud Logging and is not needed here.

### Wire the exporter

Call ADK's `maybe_set_otel_providers()` **before** importing or constructing agents. It reads the `OTEL_EXPORTER_OTLP_*` environment variables and installs HTTP OTLP exporters.

```python Python theme={null}
from google.adk.telemetry.setup import maybe_set_otel_providers
from opentelemetry.instrumentation.google_genai import GoogleGenAiSdkInstrumentor

# Environment variables above must already be set.
maybe_set_otel_providers()
GoogleGenAiSdkInstrumentor().instrument()

from google.adk.agents import Agent
from google.adk.runners import Runner

def get_weather(location: str) -> str:
    """Get weather for a location."""
    return f"Sunny and 72°F in {location}"

agent = Agent(
    name="weather_agent",
    model="gemini-2.0-flash-exp",
    description="Provides weather information",
    instructions="You help users get weather information for any location.",
    tools=[get_weather],
)

async def run_conversation(user_input: str):
    runner = Runner(agent)
    async for event in runner.run_async(
        session_id="session-123",
        user_id="user-456",
        message=user_input,
    ):
        if event.is_final_response():
            print(event.content)

await run_conversation("What's the weather in San Francisco?")
```

If you already configure a global `TracerProvider` (for example via `opentelemetry-instrument` or an in-process collector), skip `maybe_set_otel_providers()` — ADK will use the provider that is already set. If you call Vertex AI through the `vertexai` SDK rather than `google.genai`, also run `VertexAIInstrumentor().instrument()` from `opentelemetry.instrumentation.vertexai`.

To send traces through an existing OpenTelemetry Collector instead of exporting from the app, keep ADK pointed at the collector and add an OTLP/HTTP exporter on the collector to `https://api.openlayer.com/v1/otel/v1/traces` with the same `Authorization` and `x-bt-parent` headers.

### `adk web` / `adk api_server`

You do not need Google's `--otel_to_cloud` flag. That flag wires GCP exporters specifically. For an arbitrary OTLP backend, set the environment variables above and run:

```bash theme={null}
adk web path/to/your/agents_dir
```

ADK picks up `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` (or `OTEL_EXPORTER_OTLP_ENDPOINT`) and exports spans over OTLP HTTP.

### What the traces look like

A typical ADK OTel trace is hierarchical, not flat:

| Span                        | Openlayer step type | What it captures                                   |
| --------------------------- | ------------------- | -------------------------------------------------- |
| `invoke_agent {agent.name}` | Agent               | Agent name, description, conversation / session id |
| `generate_content {model}`  | Chat completion     | Model, tokens, prompts / responses                 |
| `execute_tool {tool.name}`  | Tool                | Tool name, description, arguments, result          |

Sub-agents appear as nested `invoke_agent` spans under the parent agent. That maps onto Openlayer's agent-trace hierarchy the same way other GenAI OTel sources do.

## Don't combine both paths

The Openlayer tracer replaces ADK's own OpenTelemetry tracer with a no-op so ADK does not emit a second copy of agent and tool spans. The Google GenAI / Vertex AI instrumentors patch the model SDK independently, so LLM spans can still be duplicated.

Enable **either** `init()` **or** ADK's OTel exporters pointed at Openlayer — not both. `init()` already calls `trace_google_adk()`, so running it together with Path 2 duplicates traces in Openlayer. If you switch from the native tracer to OTel, call `unpatch_google_adk()` (or restart the process without `init()`).

## Comparison: which path produces what

|                                   | Openlayer tracer                             | ADK OpenTelemetry                                                                    |
| --------------------------------- | -------------------------------------------- | ------------------------------------------------------------------------------------ |
| Setup                             | One call                                     | Env vars + `maybe_set_otel_providers()`                                              |
| Vendor SDK in app code            | Yes (`openlayer`)                            | No                                                                                   |
| Agent hierarchy                   | ✅                                            | ✅ (`invoke_agent` parent spans)                                                      |
| Tool calls                        | ✅                                            | ✅ (`execute_tool`)                                                                   |
| Tokens                            | ✅                                            | ✅ (`gen_ai.usage.*`)                                                                 |
| Prompt / response content         | ✅                                            | ✅ with `SPAN_ONLY` + `ADK_CAPTURE_MESSAGE_CONTENT_IN_SPANS=true`; ❌ if either is off |
| Agent instructions / descriptions | ✅                                            | ✅ (`gen_ai.agent.*`)                                                                 |
| Session / user ids                | ✅                                            | ⚠️ conversation id on agent spans; user id depends on ADK version                    |
| Cost                              | Estimated by Openlayer from tokens and model | Estimated the same way when tokens and model are present                             |
| Portable to other OTel backends   | ❌                                            | ✅                                                                                    |

## Development

In [development mode](/development/overview), Openlayer becomes a step in your CI/CD pipeline, and your tests get automatically evaluated after being triggered by some events.

Openlayer tests often rely on your AI system's outputs on a validation dataset. As discussed in the [Configuring output generation guide](/development/configuring-output-generation), you have two options:

1. either provide a way for Openlayer to run your AI system on your datasets, or
2. before pushing, generate the model outputs yourself and push them alongside your artifacts.

For AI systems built with Google ADK, if you are **not** computing your system's outputs yourself, you must provide your **API credentials** for Google's Gemini models.

To do so, navigate to "**Workspace settings**" -> "**Environment variables**," and click on "Add secret" to add the required Google API credentials (such as `GOOGLE_API_KEY` or appropriate service account credentials).

If you don't add the required API credentials, you'll encounter a "Missing API key" error when Openlayer tries to run your AI system to get its outputs.