Shadow Traffic

Mirror sampled requests to a secondary target without using the shadow response for the client result.

Always confirm availability in the release bundle selected for deployment.

At a glance

Property	Value
Pipeline phase	Request + response
Category	Traffic
Canonical minimum tier	Pro
Canonical entitlement	Yes
Supported deployment contract	Yes
Release status	Catalog component.

Release accuracy

• Current documentation: Catalog component.

Where any detail below conflicts with the release status above, the release status is authoritative. Field names and examples describe the current dashboard and CRD surface; always confirm behavior against the selected release bundle before relying on it operationally.

Release-aware feature flow. The diagram is explanatory; the release status on this page is authoritative.

Diagram resources: Open the SVG full screen · Download the editable Excalidraw source

Performance impact

:::warning Unverified performance figures

These figures are illustrative only. They are not current benchmarks or service guarantees and have not been verified by the current test suite.

:::

Percentile	Reported figure
P50	0.50ms
P95	1.20ms
P99	2.50ms

Note: Mirroring is asynchronous. Diff comparison and metrics run off the client path, while health checks add only in-memory bookkeeping.

Overview

The Shadow Traffic step mirrors live requests to one or more shadow backends without affecting the primary response path. It supports request sampling, bounded concurrency, connection pooling, optional primary-vs-shadow response hashing, per-target latency tracking, conditional shadow filters, and health-based auto-disable protection.

Use it when validating migrations, comparing cluster behavior, or measuring how alternate backends behave under real workloads.

When to use

• Validating a new MongoDB cluster or proxy configuration with production-like traffic
• Comparing deterministic query responses between primary and shadow backends
• Collecting latency baselines for candidate backends before a cutover
• Restricting shadowing to specific commands, databases, or collections during phased rollouts

How it works

1. Request phase: the step checks sampling, optional database/collection/command conditions, and the health state of each shadow target.
2. Dispatch: matching requests are copied (independent buffer) and submitted to a bounded worker pool (default: 128 workers, 256 queue depth). If the pool is saturated, shadow requests are silently dropped — the primary path is never blocked or slowed.
3. Response capture (optional): when diff_comparison.enabled is true, the primary response hash is stored in StepContext during the response phase so request-side shadow workers can compare results.
4. Metrics (optional): when latency_metrics.enabled is true, each target records request count, average latency, and a rolling P99 latency. Dropped tasks are also counted via nexo_shadow_dropped_total.
5. Health monitor (optional): Business tier health tracking computes recent error rate per target and temporarily skips unhealthy shadow targets until the monitoring window expires.
6. Backpressure: when shadow targets are slow or unresponsive, the bounded pool prevents goroutine accumulation and memory exhaustion. Shadow is always best-effort — degraded shadow targets never impact primary request latency or availability.

Configuration

“Not specified” means required semantics were not declared for that field.

Field	Type	Default	Required	Description
targets	object[]	—	Yes	[{ name, address, tls? }] shadow destinations that receive mirrored traffic
sample_rate	number	1.0	No	Fraction of requests to mirror from 0.0 to 1.0
max_concurrent	number	100	No	Maximum number of in-flight shadow requests across all targets
timeout_ms	number	5000	No	Dial/read/write timeout for shadow target operations
diff_comparison	object	—	No	{ enabled: bool, log_diffs: bool } (Pro)
latency_metrics	object	—	No	{ enabled: bool } (Pro)
conditions	object	—	No	{ databases?: string[], collections?: string[], commands?: string[] } (Business)
health_monitor	object	—	No	{ enabled: bool, error_threshold_pct: number, window_sec: number } (Business)

Settings reference

No additional settings reference is documented for this component.

Examples

Pro response diffing against a validation cluster

request_steps:
  - name: builtin:shadow
    config:
      targets:
        - name: validation
          address: mongo-validation.internal:27017
      sample_rate: 1.0
      max_concurrent: 200
      diff_comparison:
        enabled: true
        log_diffs: true
      latency_metrics:
        enabled: true
response_steps:
  - name: builtin:shadow
    config:
      targets:
        - name: validation
          address: mongo-validation.internal:27017
      diff_comparison:
        enabled: true
        log_diffs: true

Business conditional shadowing with health protection

request_steps:
  - name: builtin:shadow
    config:
      targets:
        - name: eu-shadow
          address: mongo-shadow-eu.internal:27017
      sample_rate: 0.25
      conditions:
        databases: [app]
        collections: [tenant_users, tenant_orders]
        commands: [find, aggregate]
      health_monitor:
        enabled: true
        error_threshold_pct: 25
        window_sec: 120

Best practices

• Add the step to both request_steps and response_steps when using diff_comparison so the primary response hash is captured.
• Start with a low sample_rate when shadowing a backend with smaller capacity.
• Use conditions to limit shadow traffic to deterministic reads when comparing responses.
• Watch ShadowMetrics output for rising latency or disabled targets before increasing traffic.
• Monitor nexo_shadow_dropped_total — if drops increase, the shadow target may need capacity or sample_rate reduction.

Limitations

• Shadow requests still execute writes on the shadow backend unless filtered out by conditions.
• Response diffing compares hashes, not semantic BSON differences.
• Health-based disablement is temporary and scoped to the configured monitoring window.
• When the worker pool queue is full, shadow requests are dropped (best-effort, never blocks primary).

Security and operational guidance

• Do not mirror sensitive or production traffic: the current runtime uses plaintext TCP and does not honor the shadow target TLS field
• Use only synthetic or otherwise non-sensitive test data unless an independently authenticated encrypted tunnel protects the entire shadow path
• Bound concurrency and timeouts so shadow failure cannot exhaust the Proxy

Related steps

• traffic-split
• metrics

Release availability

• Current documentation: Catalog component.

See the component catalog for the complete comparison matrix.