Skip to content

MCPToolset does not propagate trace headers if presentΒ #4768

@conorhodder

Description

@conorhodder

πŸ”΄ Required Information

Describe the Bug:
When the ADK's MCP toolset makes outbound HTTP requests to downstream MCP servers, the httpx client used does not propagate W3C traceparent headers. This means agent β†’ MCP service calls create disconnected traces, breaking distributed trace correlation between the agent and the services it calls tools on.

The root cause is that StreamableHTTPConnectionParams uses a default httpx_client_factory (create_mcp_http_client) that creates a plain httpx.AsyncClient without OpenTelemetry instrumentation. Even though the TracerProvider and TraceContextTextMapPropagator are correctly configured by the ADK, the MCP HTTP client is never instrumented, so the propagator is never invoked on outbound requests.

Steps to Reproduce:

  1. Deploy an ADK agent with an MCP toolset connecting to a downstream service over HTTP:
    campaign_tools = McpToolset(
        connection_params=StreamableHTTPConnectionParams(
            url="http://some-mcp-server:8080/mcp",
        ),
    )
  2. Set OTEL_EXPORTER_OTLP_ENDPOINT on both the agent and the downstream service
  3. Send a message that triggers the agent to call an MCP tool
  4. Observe in Jaeger that the agent trace and the downstream service trace are separate trace roots β€” the MCP HTTP request does not carry a traceparent header

Expected Behavior:
Outbound MCP HTTP requests should carry the W3C traceparent header from the current trace context, so that the downstream service creates child spans under the agent's trace, producing a single end-to-end distributed trace (e.g. web β†’ agent β†’ MCP service).

Observed Behavior:
The MCP service receives requests without a traceparent header and creates a new trace root. The agent's trace shows execute_tool get_order_details but the downstream service's POST /mcp span is in a separate, unrelated trace.

Workaround: provide a custom httpx_client_factory that instruments the client:

from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor
from google.adk.tools.mcp_tool.mcp_session_manager import create_mcp_http_client

def create_instrumented_mcp_client(headers=None, timeout=None, auth=None):
    client = create_mcp_http_client(headers=headers, timeout=timeout, auth=auth)
    HTTPXClientInstrumentor.instrument_client(client)
    return client

campaign_tools = McpToolset(
    connection_params=StreamableHTTPConnectionParams(
        url="http://campaign:8080/mcp/mcp",
        httpx_client_factory=create_instrumented_mcp_client,
    ),
)

Environment Details:

  • ADK Library Version: google-adk 1.25.0
  • Desktop OS: Linux 6.10.14-linuxkit (Docker)
  • Python Version: 3.14.3

Model Information:

  • Are you using LiteLLM: No
  • Which model is being used: gemini-2.5-flash

🟑 Optional Information

Regression:
N/A β€” create_mcp_http_client has never applied OTel instrumentation. The httpx_client_factory parameter on StreamableHTTPConnectionParams was added as an extension point but the default factory does not instrument.

Logs:

# Default MCP client factory creates a plain httpx.AsyncClient:
def create_mcp_http_client(headers=None, timeout=None, auth=None):
    kwargs = {"follow_redirects": True}
    if timeout is None:
        kwargs["timeout"] = httpx.Timeout(MCP_DEFAULT_TIMEOUT, read=MCP_DEFAULT_SSE_READ_TIMEOUT)
    else:
        kwargs["timeout"] = timeout
    if headers is not None:
        kwargs["headers"] = headers
    if auth is not None:
        kwargs["auth"] = auth
    return httpx.AsyncClient(**kwargs)
    # ^ No OTel instrumentation applied

# Meanwhile _setup_instrumentation_lib_if_installed() only instruments GoogleGenAiSdkInstrumentor,
# not httpx clients.

Suggested Fix:
In create_mcp_http_client, conditionally instrument the client when OpenTelemetry is available:

def create_mcp_http_client(headers=None, timeout=None, auth=None):
    # ... existing logic ...
    client = httpx.AsyncClient(**kwargs)

    try:
        from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor
        HTTPXClientInstrumentor.instrument_client(client)
    except ImportError:
        pass

    return client

This is consistent with the existing try/except ImportError pattern used for GoogleGenAiSdkInstrumentor in _setup_instrumentation_lib_if_installed().

How often has this issue occurred?:

  • Always (100%)

Metadata

Metadata

Assignees

Labels

tracing[Component] This issue is related to OpenTelemetry tracing

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions