Context Propagation in Distributed Systems: Rust Tracing

Context propagation is how a trace ID and related metadata follow a request across service boundaries. When Service A calls Service B via HTTP, you need Service B to continue the same trace, creating a child span linked to Service A's parent span. Without context propagation, each service sees only its own spans, and you lose the ability to debug end-to-end failures.

OpenTelemetry standardizes context propagation via the W3C Trace Context header format and baggage, making it straightforward to link traces across heterogeneous microservices.

Understanding Trace Context and Baggage

When Service A processes a request, it creates a span with a unique trace_id. If it calls Service B over HTTP, it must send:

1. Trace Context (W3C format): Standard HTTP headers (traceparent, tracestate) that contain:

   • trace_id — unique per request, shared across all services.
   • parent_id — the span ID of the calling operation.
   • trace_flags — whether this trace should be sampled.

2. Baggage (optional): Custom key-value pairs (user ID, environment, request ID) that propagate with the trace but are not part of the standard trace context.

Example HTTP request from Service A to Service B:

GET /api/order-details HTTP/1.1
Host: service-b.internal
traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01
baggage: user_id=12345, environment=production

Service B parses these headers, extracts the trace_id and parent_id, and creates a new child span with the same trace_id. Jaeger automatically stitches these spans together into one logical trace.

Extracting Context in Rust

When you receive an HTTP request, extract the trace context from headers:

use opentelemetry::global;
use opentelemetry::propagation::Extractor;
use opentelemetry_jaeger::Tracer;
use axum::http::HeaderMap;

// Implement Extractor for HeaderMap
struct HeaderExtractor<'a>(&'a HeaderMap);

impl<'a> Extractor for HeaderExtractor<'a> {
    fn get(&self, key: &str) -> Option<String> {
        self.0
            .get(key)
            .and_then(|v| v.to_str().ok())
            .map(|s| s.to_string())
    }

    fn keys(&self) -> Vec<&str> {
        self.0.keys().map(|h| h.as_str()).collect()
    }
}

#[tokio::main]
async fn main() {
    // Handler that extracts context from incoming request
    async fn handle_request(headers: HeaderMap) -> String {
        let propagator = opentelemetry_jaeger::Propagator::new();
        let extractor = HeaderExtractor(&headers);

        // Extract context from headers
        let context = propagator.extract(&extractor);

        // Create a span linked to the parent
        let tracer = global::tracer("service_b");
        let mut span = tracer.start_with_context("process_order", &context);
        span.set_attribute(opentelemetry::KeyValue::new("order.id", "12345"));

        "Order processed".to_string()
    }

    handle_request(HeaderMap::new()).await;
}

However, using the propagator API directly is verbose. A simpler approach uses tracing and tracing-opentelemetry:

Automatic Context Propagation with tracing

The tracing-opentelemetry crate integrates context propagation into the tracing ecosystem automatically:

use tracing_opentelemetry::OpenTelemetryLayer;
use tracing_subscriber::layer::SubscriberExt;
use axum::{Router, routing::post, extract::Json};
use opentelemetry_jaeger::new_pipeline;

#[tokio::main]
async fn main() {
    // Initialize Jaeger and wire to tracing
    let jaeger_tracer = new_pipeline()
        .install_simple()
        .expect("Jaeger failed");

    tracing_subscriber::registry()
        .with(OpenTelemetryLayer::new(jaeger_tracer))
        .init();

    // Handler that automatically extracts context
    async fn handle_order_request(Json(order): Json<serde_json::Value>) -> String {
        // The tracing layer automatically extracts W3C Trace Context
        // from the HTTP headers and attaches it to the current span
        tracing::info!("Processing order: {:?}", order);
        "OK".to_string()
    }

    let app = Router::new()
        .route("/api/order", post(handle_order_request));

    let listener = tokio::net::TcpListener::bind("127.0.0.1:8080").await.unwrap();
    axum::serve(listener, app).await.unwrap();
}

The OpenTelemetryLayer automatically extracts incoming trace context and links new spans to it. No manual header parsing required.

Injecting Context When Calling Other Services

When Service A calls Service B, it must inject the current trace context into the request headers:

use opentelemetry::propagation::Injector;
use opentelemetry::global;
use reqwest::Client;
use std::collections::HashMap;

// Implement Injector for HashMap
struct HashMapInjector(HashMap<String, String>);

impl Injector for HashMapInjector {
    fn set(&mut self, key: &str, value: String) {
        self.0.insert(key.to_string(), value);
    }
}

#[instrument(name = "call_service_b")]
async fn fetch_user_from_service_b(user_id: u64) -> Result<String> {
    // Create an injector to hold headers
    let mut injector = HashMapInjector(HashMap::new());

    // Inject the current trace context
    let propagator = opentelemetry_jaeger::Propagator::new();
    propagator.inject_context(&tracing::Span::current(), &mut injector);

    // Build the HTTP request with injected headers
    let client = Client::new();
    let mut request = client
        .get(&format!("http://service-b.internal/api/user/{}", user_id));

    for (key, value) in injector.0.iter() {
        request = request.header(key, value);
    }

    let response = request.send().await?;
    Ok(response.text().await?)
}

Alternatively, use reqwest's built-in OpenTelemetry integration if available, or middleware that handles injection automatically.

Baggage: Custom Metadata Propagation

Baggage carries custom key-value pairs across services. Common examples: user ID, request ID, feature flags, environment.

use opentelemetry::baggage::Baggage;

#[instrument(name = "process_request")]
async fn handle_request(user_id: u64) {
    // Set baggage
    let mut baggage = Baggage::new();
    baggage = baggage.with_kv("user_id", user_id.to_string());
    baggage = baggage.with_kv("feature_flag", "new_checkout");

    // This baggage is automatically injected into headers when you make
    // HTTP calls to other services (if using tracing-opentelemetry)

    tracing::info!("Baggage set");

    // Call another service (baggage propagates automatically)
    call_payment_service().await;
}

#[instrument(name = "call_payment_service")]
async fn call_payment_service() -> Result<()> {
    // In the payment service, extract baggage:
    // let baggage = opentelemetry::baggage::get_active();
    // let user_id = baggage.get_str("user_id");
    Ok(())
}

Context Propagation in HTTP Clients and Servers

For HTTP servers (receiving requests):

use axum::{Router, routing::get};
use tower_http::trace::TraceLayer;
use opentelemetry_http_compat::SdkHttpTracer;

let app = Router::new()
    .route("/api/users", get(handler))
    .layer(
        TraceLayer::new_for_http()
            .on_request(DefaultOnRequest::new().with_tags(Vec::new()))
    );

For HTTP clients (making requests):

use reqwest::Client;
use reqwest_opentelemetry::OpenTelemetryClientLayer;
use http_body_util::Full;

let client = Client::builder()
    .layer(OpenTelemetryClientLayer)
    .build()
    .expect("Client failed");

let response = client.get("http://service-b/api/data").send().await?;

These automatically inject and extract W3C Trace Context headers.

Testing Context Propagation

To verify context propagation works:

1. Start Service A and Service B, both with Jaeger export.
2. Call an endpoint on Service A that invokes Service B.
3. Open Jaeger UI (http://localhost:16686).
4. Search for your trace by trace ID.
5. You should see both services' spans in a single trace with proper parent-child relationships.

Example output in Jaeger:

service-a: handle_request (100ms)
├─ service-a: call_service_b (80ms)
│  └─ [HTTP request sent with traceparent header]
│     └─ service-b: fetch_resource (75ms)
│        └─ service-b: database_query (50ms)

Key Takeaways

• Context propagation sends trace ID and parent ID across service boundaries via HTTP headers.
• W3C Trace Context (traceparent header) is the standard format; OpenTelemetry implements it automatically.
• Extract context from incoming requests; inject context into outgoing requests.
• Baggage carries custom metadata (user ID, feature flags) across services.
• The tracing-opentelemetry crate handles context extraction/injection automatically if integrated properly.

Frequently Asked Questions

Does context propagation work with gRPC?

Yes. gRPC uses similar metadata propagation but via gRPC metadata instead of HTTP headers. OpenTelemetry provides propagators for gRPC. The pattern is identical: extract context from metadata on receive, inject into metadata when calling.

What if a service doesn't support OpenTelemetry?

The trace breaks. You will see spans in Service A and Service B separately, but not linked. Consider deploying a tracing proxy or sidecar (e.g., Envoy) that injects/extracts context transparently.

How much overhead does context propagation add?

Negligible. Extracting and injecting headers is microseconds. The overhead is in span creation and export, not propagation.

Can I include sensitive data in baggage?

Technically yes, but you should not. Baggage is logged and transmitted across services. Use only non-sensitive metadata (request IDs, feature flags, environment). For sensitive data, store it in logs (which may be encrypted).

What if I am in a monolithic service without inter-service calls?

You still benefit from context propagation internally. Spans within the same service are linked to the same trace, and you can trace through function calls. Context propagation only becomes critical when you have multiple services.

Further Reading

• W3C Trace Context Specification
• OpenTelemetry Propagators
• Baggage Specification
• Distributed Tracing Best Practices