Shadow Traffic

Mirror sampled requests to a secondary target without using the shadow response for the client result.

This page belongs to the immutable 0.2.0 Private Preview documentation.

At a glance

Property	Value
Pipeline phase	Request + response
Category	Traffic
Canonical minimum tier	Pro
Legacy dashboard tier label	Pro
Legacy rendered name	Shadow Traffic
Legacy rendered summary	Async request mirroring with optional response diffing, latency metrics, and business-grade targeting safeguards.
Canonical entitlement	Yes
Supported deployment contract	Yes
Release status	Reconciled and executable.

Release accuracy

• 0.2.0 Private Preview: Reconciled and executable.

The detailed material below preserves every section rendered by the legacy dashboard. Where it conflicts with the release status above, the release status is authoritative. Legacy field names and examples are not a substitute for the selected bundle's CRD and runtime contract. Unsafe legacy wording is retained in metadata for traceability but is corrected in the rendered guidance.

Release-aware feature flow. The diagram is explanatory; the release status on this page is authoritative.

Diagram resources: Open the SVG full screen · Download the editable Excalidraw source

Performance impact

:::warning Legacy, unverified performance claims

These numbers are preserved for documentation parity with the legacy dashboard. They are not current benchmarks or service guarantees and have not been verified by the current test suite.

:::

Percentile	Legacy claim
P50	0.50ms
P95	1.20ms
P99	2.50ms

Legacy note: Mirroring is asynchronous. Diff comparison and metrics run off the client path, while health checks add only in-memory bookkeeping.

Overview

The Shadow Traffic step mirrors live requests to one or more shadow backends without affecting the primary response path. It supports request sampling, bounded concurrency, connection pooling, optional primary-vs-shadow response hashing, per-target latency tracking, conditional shadow filters, and health-based auto-disable protection.

Use it when validating migrations, comparing cluster behavior, or measuring how alternate backends behave under real workloads.

When to use

• Validating a new MongoDB cluster or proxy configuration with production-like traffic
• Comparing deterministic query responses between primary and shadow backends
• Collecting latency baselines for candidate backends before a cutover
• Restricting shadowing to specific commands, databases, or collections during phased rollouts

How it works

1. Request phase: the step checks sampling, optional database/collection/command conditions, and the health state of each shadow target.
2. Dispatch: matching requests are copied (independent buffer) and submitted to a bounded worker pool (default: 128 workers, 256 queue depth). If the pool is saturated, shadow requests are silently dropped — the primary path is never blocked or slowed.
3. Response capture (optional): when diff_comparison.enabled is true, the primary response hash is stored in StepContext during the response phase so request-side shadow workers can compare results.
4. Metrics (optional): when latency_metrics.enabled is true, each target records request count, average latency, and a rolling P99 latency. Dropped tasks are also counted via nexo_shadow_dropped_total.
5. Health monitor (optional): Business tier health tracking computes recent error rate per target and temporarily skips unhealthy shadow targets until the monitoring window expires.
6. Backpressure: when shadow targets are slow or unresponsive, the bounded pool prevents goroutine accumulation and memory exhaustion. Shadow is always best-effort — degraded shadow targets never impact primary request latency or availability.

Configuration

The table preserves the legacy dashboard field reference. “Not specified” means the legacy source did not declare required semantics.

Field	Legacy UI type	Legacy default	Required	Description
targets	object[]	—	Yes	[{ name, address, tls? }] shadow destinations that receive mirrored traffic
sample_rate	number	1.0	No	Fraction of requests to mirror from 0.0 to 1.0
max_concurrent	number	100	No	Maximum number of in-flight shadow requests across all targets
timeout_ms	number	5000	No	Dial/read/write timeout for shadow target operations
diff_comparison	object	—	No	{ enabled: bool, log_diffs: bool } (Pro)
latency_metrics	object	—	No	{ enabled: bool } (Pro)
conditions	object	—	No	{ databases?: string[], collections?: string[], commands?: string[] } (Business)
health_monitor	object	—	No	{ enabled: bool, error_threshold_pct: number, window_sec: number } (Business)

Settings reference

The legacy page did not render an additional anchored settings reference.

Examples

Pro response diffing against a validation cluster

request_steps:
  - name: builtin:shadow
    config:
      targets:
        - name: validation
          address: mongo-validation.internal:27017
      sample_rate: 1.0
      max_concurrent: 200
      diff_comparison:
        enabled: true
        log_diffs: true
      latency_metrics:
        enabled: true
response_steps:
  - name: builtin:shadow
    config:
      targets:
        - name: validation
          address: mongo-validation.internal:27017
      diff_comparison:
        enabled: true
        log_diffs: true

Business conditional shadowing with health protection

request_steps:
  - name: builtin:shadow
    config:
      targets:
        - name: eu-shadow
          address: mongo-shadow-eu.internal:27017
      sample_rate: 0.25
      conditions:
        databases: [app]
        collections: [tenant_users, tenant_orders]
        commands: [find, aggregate]
      health_monitor:
        enabled: true
        error_threshold_pct: 25
        window_sec: 120

Best practices

• Add the step to both request_steps and response_steps when using diff_comparison so the primary response hash is captured.
• Start with a low sample_rate when shadowing a backend with smaller capacity.
• Use conditions to limit shadow traffic to deterministic reads when comparing responses.
• Watch ShadowMetrics output for rising latency or disabled targets before increasing traffic.
• Monitor nexo_shadow_dropped_total — if drops increase, the shadow target may need capacity or sample_rate reduction.

Limitations

• Shadow requests still execute writes on the shadow backend unless filtered out by conditions.
• Response diffing compares hashes, not semantic BSON differences.
• Health-based disablement is temporary and scoped to the configured monitoring window.
• When the worker pool queue is full, shadow requests are dropped (best-effort, never blocks primary).

Security and operational guidance

• Do not mirror sensitive or production traffic in 0.2.0: the runtime uses plaintext TCP and does not honor the shadow TLS field
• Use only synthetic or otherwise non-sensitive test data
• Bound concurrency and timeouts so shadow failure cannot exhaust the Proxy

Related steps

• traffic-split
• metrics

Release availability

• 0.2.0 Private Preview: Reconciled and executable.

See the component catalog for the complete comparison matrix.