Pipeline Builder workflow

Pipeline Builder is the visual editing surface for Nexo policy pipelines. The editor auto-saves pipeline state, but edits do not change the active data path until validation passes and the user completes the Deploy workflow.

Build a draft

1. Open the target project and environment.
2. Create a pipeline or open an existing pipeline.
3. Select supported components from the catalog.
4. Place components in the applicable request, response, router, or connection phase.
5. Configure each component and resolve validation warnings.
6. Review the generated Kubernetes resources in the deployment dialog.

The available catalog depends on entitlement, protocol, and installed release. A visible legacy or sample card is not evidence that the installed bundle can deploy it.

Ordering rules

Request components execute in their listed order before the upstream operation. Response components execute in their listed order after the upstream response. A component that rejects an operation terminates that path.

Practical ordering principles:

• Do not rely on 0.2.0 Access Control: its Operator and Proxy configuration contracts differ. Use database-native authorization and independently verified controls.
• Put effective hard request limits before expensive transformation or observation.
• Establish authenticated tenant identity before tenant-aware rate limits, cache keys, routing, or masking policy.
• Apply request rewrites before components that must evaluate the rewritten value.
• Apply response masking before logging or audit sinks that could capture protected fields.
• Keep metrics and low-risk aggregate observation near the end of each applicable path.

There is no universal safe order. Validate administrative commands, reads, writes, transactions, errors, and large BSON messages for the intended sequence.

Validation

Builder validation can detect missing required fields, invalid references, phase mismatch, unsupported combinations, and release-specific availability. Server-side and Operator validation remain authoritative; a draft that renders in the browser can still be rejected before activation.

Deploy and verify

Deploying applies the generated desired state through the supported control path:

1. Review the generated CRDs and deployment target.
2. Confirm Deploy through the supported console workflow.
3. Wait for Manager and Operator acknowledgement.
4. Confirm the active graph revision and runtime health.
5. Run representative connectivity and policy checks.

Import or export is a configuration transfer mechanism, not proof that the target environment supports every referenced component. Use the release and configuration rollback surfaces explicitly published by the installed bundle.

0.2.0 boundaries

The 0.2.0 Proxy renders router and connection phases but does not execute them. Several installed CRD schemas are also not registered by the pinned Operator. Use the component catalog and Operator CRD catalog before deploying a 0.2.0 pipeline.

Preserved legacy operations reference

:::danger Unavailable in Nexo 0.2.0

This frozen section preserves every legacy workflow, command, flag, example, table, troubleshooting item, limitation, and related link for parity. It does not make those interfaces available in 0.2.0. The 0.2.0 Private Preview has no public Docker/Compose or VM/binary distribution, no public chart repository, no supported end-to-end nexoctl login/status/upgrade workflow, no profiling API or profiling CLI, no canary control API, no automatic promotion/rollback engine, and no executable router or connection phase. Every shell command below is historical, not a copy/paste procedure. Never pipe an unverified download into a shell, consume a mutable latest artifact, or place connect tokens, license JWTs, passwords, or API keys in process arguments. Require a pinned artifact plus an approved signature/checksum and the licensed secret-delivery workflow. Use placeholders only and follow the supported 0.2.0 guidance above this section.

:::

Legacy UI labels:

• Drag & Drop
• Real-time Cost
• One-Click Deploy

Overview

The pipeline builder is a visual editor in the Nexo dashboard that lets you design request/response pipelines without writing configuration files. Drag steps from the catalog, configure them, preview the cost, and deploy — all from a single screen.

Every Nexo proxy runs a pipeline: an ordered sequence of steps that intercept, inspect, transform, or route MongoDB wire-protocol traffic. The builder gives you a graphical canvas with two vertical lanes — Request and Response — so you can see exactly what happens to traffic in each direction.

Changes made in the builder are saved as draft configurations. Nothing goes live until you explicitly click Deploy. This means you can experiment freely, preview the generated YAML/JSON, check costs, and share the draft with your team before any traffic is affected. The builder combines a step catalog on the left with request and response lanes in the center, plus inline cost and deployment controls.

Step Catalog

The left sidebar displays every step available for your pipeline. Steps are organized into collapsible categories and include both built-in steps and marketplace plugins.

Categories

• Observability — Logging, metrics collection, distributed tracing, and audit trails.
• Security — Authentication, field-level encryption, IP allow-lists, and query sanitization.
• Traffic — Rate limiting, query routing, read/write splitting, and load balancing.
• Performance — Response caching, connection pooling, query optimization hints, and compression.
• Compliance — Data masking, PII detection, GDPR right-to-erasure hooks, and SOC 2 audit logging.

Tier Filtering

Steps that require a higher tier than your current subscription are shown grayed out with an "Upgrade" badge. Hovering over them shows which tier is needed and a quick link to the billing page.

Search & Filter

A search bar at the top of the catalog lets you filter steps by name or keyword. You can also toggle filters to show only built-in steps, only marketplace plugins, or only steps available on your current tier.

Marketplace Plugins

Plugins installed from the Nexo Marketplace appear alongside built-in steps in the catalog. They are marked with a plugin badge and display their per-month pricing. Clicking a marketplace step that you haven't installed yet takes you to its marketplace listing page.

Step Card Anatomy

Each step in the catalog is rendered as a draggable card showing:

• Name — e.g., "Rate Limiter"
• Description — one-line summary of what the step does
• Tier Badge — Free, Pro, Business, or Enterprise
• Category Icon — visual indicator of the step's category

// Example step catalog entry (internal schema)
{
  "id": "rate-limiter",
  "name": "Rate Limiter",
  "description": "Limit requests per client IP or API key",
  "category": "Traffic",
  "tier": "Business",
  "icon": "shield-check",
  "config_schema": {
    "type": "object",
    "properties": {
      "max_requests": { "type": "number", "default": 1000 },
      "window_seconds": { "type": "number", "default": 60 },
      "key_by": { "type": "string", "enum": ["ip", "api_key", "user_id"] }
    },
    "required": ["max_requests", "window_seconds"]
  }
}

Building a Pipeline

Constructing a pipeline is as simple as dragging steps from the catalog onto the canvas. The builder enforces ordering rules and warns you about potential conflicts.

Drag & Drop

Grab a step card from the catalog sidebar and drop it onto either the Request lane or the Response lane. A blue drop-zone indicator shows you exactly where the step will be inserted.

Execution Order

Steps execute top-to-bottom in the order shown on the canvas. The first step in the Request lane runs first when a new MongoDB operation arrives; the last step in the Response lane runs just before the response is sent back to the client.

Reordering

Click and drag any step within its lane to change its position. A ghost preview shows where the step will land when released. You can also use the arrow buttons that appear on hover to nudge a step up or down by one position.

Dependency Warnings

Some steps have soft dependencies on other steps. The builder shows an amber warning icon when a dependency recommendation is not met. For example:

• "Metrics" should come after "Logging" for accurate timing — placing metrics before logging may yield incorrect latency numbers.
• "Compression" should be the last step in the Response lane — compressing before other response steps means those steps decompress and re-compress, hurting performance.
• "Auth" should be the first step in the Request lane — running other steps before authentication means unauthenticated traffic is processed unnecessarily.

Incompatible Step Warnings

Certain steps are mutually exclusive. The builder shows a red error when you try to add an incompatible combination:

• Two Router steps in the same lane — only one routing decision can be made per request.
• Two Rate Limiter steps with the same key_by field — use a single rate limiter with multiple rules instead.
• A Cache step and a Real-time Analytics step — cached responses bypass analytics, leading to inaccurate data.

# Example pipeline structure (YAML)
pipeline:
  request:
    - step: auth
      config:
        provider: jwt
        jwks_url: https://auth.example.com/.well-known/jwks.json
    - step: logging
      config:
        level: info
        include_body: false
    - step: rate-limiter
      config:
        max_requests: 1000
        window_seconds: 60
        key_by: api_key
    - step: filter
      config:
        deny_collections: ["internal_*", "system.*"]
  response:
    - step: metrics
      config:
        export_to: prometheus
        histogram_buckets: [0.01, 0.05, 0.1, 0.5, 1, 5]
    - step: compression
      config:
        algorithm: zstd
        min_size_bytes: 1024

Configuring Steps

Clicking any step on the canvas opens its configuration panel on the right side of the screen. The form fields are auto-generated from the step's config_schema (JSON Schema), so every step has a tailored, validated editing experience.

Field Types

The builder supports the following form field types, mapped from JSON Schema types:

• Text — free-form string input for URLs, names, patterns, and other textual values.
• Number — numeric input with optional min/max constraints, step increments, and unit labels (e.g., "ms", "requests").
• Boolean — toggle switch for on/off settings like "include_body" or "enable_tls".
• Select — dropdown for enum values such as log levels (debug, info, warn, error).
• Password — masked input for secrets like API keys or tokens. Values are stored as Nexo secret references, never in plain text.

Validation

The configuration panel validates inputs in real-time as you type:

• Required fields — highlighted with a red border if left empty.
• Format validation — URLs, email addresses, regex patterns, and CIDR ranges are checked for well-formedness.
• Range checks — numeric fields enforce minimum and maximum bounds (e.g., rate limit window must be ≥ 1 second).
• Cross-field validation — some fields depend on others (e.g., window_seconds must be greater than burst_window).

Example: Logging Step

// Logging step configuration
{
  "step": "logging",
  "config": {
    "level": "info",              // select: debug | info | warn | error
    "format": "json",             // select: json | text | ecs
    "include_body": false,        // boolean toggle
    "include_headers": true,      // boolean toggle
    "max_body_size_bytes": 4096,  // number (min: 0, max: 65536)
    "redact_fields": [            // text array
      "password",
      "credit_card"
    ],
    "output": "stdout",           // select: stdout | file | syslog
    "sample_rate": 1.0            // number (min: 0.0, max: 1.0, step: 0.01)
  }
}

Example: Filter Step

// Filter step configuration
{
  "step": "filter",
  "config": {
    "mode": "deny",                  // select: allow | deny
    "deny_collections": [            // text array — glob patterns supported
      "internal_*",
      "system.*",
      "_migrations"
    ],
    "deny_operations": [             // select (multi): find | insert | update | delete | aggregate
      "delete"
    ],
    "max_document_size_bytes": 16777216,  // number (default: 16 MB)
    "reject_message": "Operation not permitted by pipeline policy"
  }
}

Example: Rate-Limit Step

// Rate-limit step configuration
{
  "step": "rate-limiter",
  "config": {
    "max_requests": 1000,           // number (min: 1)
    "window_seconds": 60,           // number (min: 1)
    "key_by": "api_key",            // select: ip | api_key | user_id | connection
    "burst_allowed": 50,            // number (min: 0)
    "burst_window_seconds": 5,      // number (min: 1)
    "on_limit": "reject",           // select: reject | queue | throttle
    "reject_status_code": 429,      // number (read-only, informational)
    "include_retry_after": true,    // boolean toggle
    "shared_across_proxies": false  // boolean toggle — requires Redis backend
  }
}

Cost Estimation

A sticky bottom bar displays the real-time estimated monthly cost for your pipeline configuration. The cost updates instantly as you add, remove, or reconfigure steps.

How Cost Is Calculated

• Base tier cost — your subscription tier's monthly fee (Free: $0, Pro: $79, Growth: $179, Business: $249, Enterprise: custom starting at $1,500/mo).
• Per-step additions — most built-in steps are included in your tier at no extra charge. Steps that require a higher tier show the upgrade cost.
• Marketplace plugin costs — each marketplace plugin has its own monthly price, shown separately in the breakdown.
• Per-proxy pricing — the total is shown per proxy instance. If you deploy to multiple proxies, multiply accordingly.

Cost Breakdown Example

The bottom bar expands to show a detailed breakdown when clicked: Example breakdown: base Pro tier pricing, upgrade requirements for gated steps, individual marketplace plugin charges, and an estimated per-proxy monthly total. If any step requires an upgrade, a banner appears at the top of the canvas explaining what tier is needed and linking to the billing page. You can still design and save your pipeline as a draft, but deployment is blocked until the upgrade is completed.

Multi-Proxy Cost Preview

When you select multiple target proxies in the deploy modal, the cost bar updates to show the aggregate monthly cost. For example, deploying to 3 proxy instances at $79.00/mo each displays a total of $237.00/mo.

Pipeline Preview

Before deploying, you can inspect the raw configuration that the builder generates. The preview panel helps you verify that the visual canvas maps correctly to the underlying config.

JSON / YAML Toggle

Click the toggle at the top of the preview panel to switch between JSON and YAML representations. Both are semantically identical — choose whichever you find more readable. The selected format is remembered across sessions.

// JSON preview
{
  "pipeline": {
    "request": [
      {
        "step": "auth",
        "config": { "provider": "jwt", "jwks_url": "https://auth.example.com/.well-known/jwks.json" }
      },
      {
        "step": "logging",
        "config": { "level": "info", "include_body": false }
      },
      {
        "step": "rate-limiter",
        "config": { "max_requests": 1000, "window_seconds": 60, "key_by": "api_key" }
      }
    ],
    "response": [
      {
        "step": "metrics",
        "config": { "export_to": "prometheus" }
      },
      {
        "step": "compression",
        "config": { "algorithm": "zstd", "min_size_bytes": 1024 }
      }
    ]
  }
}

Diff View

When modifying an existing deployed pipeline, the preview panel shows a side-by-side diff highlighting what changed. Added lines are shown in green, removed lines in red, and unchanged lines in gray. This makes it easy to review exactly what will change when you deploy.

  pipeline:
    request:
      - step: auth
        config:
          provider: jwt
-     - step: logging
-       config:
-         level: debug
-         include_body: true
+     - step: logging
+       config:
+         level: info
+         include_body: false
+     - step: rate-limiter
+       config:
+         max_requests: 1000
+         window_seconds: 60

Validation Checks

The preview panel includes a validation checklist showing the status of your configuration:

• ✅ Schema valid — all step configurations pass JSON Schema validation.
• ✅ No conflicts — no incompatible step combinations detected.
• ✅ Dependencies met — all soft dependency recommendations are satisfied.
• ✅ Tier compatible — all steps are available on your current tier.
• ✅ Secrets resolved — all secret references point to existing Nexo secrets.

Any failing check is shown with a ❌ icon and a description of the issue. You cannot deploy a pipeline with failing validation checks (except tier warnings, which are treated as blocking only at deploy time).

Deploying

When your pipeline is ready, click the Deploy button in the bottom bar. This opens the deployment modal where you confirm your choices and select a deployment strategy.

Confirmation Modal

The deploy modal summarizes everything about the deployment:

• Cost — the estimated monthly cost per proxy and total cost across all targeted proxies.
• Target clusters — which clusters will receive the new pipeline.
• Affected proxies — a list of proxy instances that will be updated, with their current pipeline version.
• Change summary — a concise diff of what is changing relative to the currently deployed pipeline.

Deploy Options

Choose one of three deployment strategies: ⚡ Immediate Push the new pipeline to all targeted proxies immediately. Each proxy reloads its configuration within seconds. Best for development environments or urgent fixes. 🐤 Canary Deploy the pipeline to a percentage of traffic first. Monitor error rates and latency before promoting to 100%. Recommended for production changes. 🕐 Scheduled Schedule the deployment for a specific date and time. Useful for coordinating changes during maintenance windows or off-peak hours.

Deployment Progress

After confirming, the modal transitions to a live progress view showing real-time deployment status for each cluster and proxy:

Deployment #d-20250114-001
Pipeline: production-observability (v3 → v4)
Strategy: Immediate

Cluster: us-east-1
  ✅ proxy-east-1a    Updated (2s)
  ✅ proxy-east-1b    Updated (3s)
  🔄 proxy-east-1c    Updating...

Cluster: eu-west-1
  ⏳ proxy-west-1a    Pending
  ⏳ proxy-west-1b    Pending

Overall: 2/5 proxies updated

Rollback

Every deployment is versioned. If something goes wrong, click Rollback to instantly revert all targeted proxies to the previously deployed pipeline version. The rollback button is available on the deployment detail page and in the pipeline builder's version history panel.

Rollbacks use the same deployment strategies — you can do an immediate rollback or a canary rollback if you want to be cautious.

Canary Deployment Option

Canary deployments let you validate pipeline changes with a small slice of production traffic before rolling out to everyone. This is the recommended strategy for all production changes.

Configuration

Toggle "Deploy as Canary" in the deploy modal to enable canary mode. Then configure the following:

• Traffic percentage — the percentage of traffic routed to the new pipeline (default: 5%).
• Rollback thresholds — automatic rollback triggers: error rate increase (> X%), latency delta (> Y ms).
• Auto-promote rules — conditions for automatically promoting the canary to 100%: minimum duration (e.g., 30 minutes), minimum number of requests (e.g., 10,000).

// Canary deployment configuration
{
  "strategy": "canary",
  "canary": {
    "traffic_percentage": 5,
    "rollback_thresholds": {
      "error_rate_increase_pct": 2.0,
      "latency_delta_ms": 50,
      "p99_latency_ms": 500
    },
    "auto_promote": {
      "enabled": true,
      "min_duration_minutes": 30,
      "min_requests": 10000,
      "max_error_rate_pct": 0.5
    },
    "monitoring": {
      "dashboard_url": "/canary/{deployment_id}",
      "alerting_channel": "#nexo-deployments"
    }
  }
}

Monitoring

While a canary is active, the deployment page shows a live comparison between the canary and the baseline:

Canary Deployment #d-20250114-002
Status: Active — 5% traffic

                  Baseline (v3)    Canary (v4)
  Error Rate:     0.12%            0.14%        ✅ Within threshold
  p50 Latency:    12ms             13ms         ✅ Within threshold
  p99 Latency:    89ms             94ms         ✅ Within threshold
  Requests:       194,230          10,221       ⏳ Need 10,000 (met!)
  Duration:       18 min           —            ⏳ Need 30 min

  Auto-promote in: ~12 minutes

For a deep dive into canary deployments, see the Canary Deployments documentation page.

Pipeline Templates

Templates are pre-built pipeline configurations for common use cases. They give you a solid starting point that you can customize to match your specific requirements.

Available Templates

📊 Production Observability Full observability stack: structured logging, Prometheus metrics, and distributed tracing with OpenTelemetry.

Steps: Auth → Logging → Tracing → Filter | Metrics → Compression 🔒 Security Hardening Defense-in-depth: JWT authentication, rate limiting, IP allow-listing, query sanitization, and field-level encryption.

Steps: Auth → IP Allow-List → Rate Limiter → Query Sanitizer → Encryption | Audit Log 🐛 Development Debug Verbose logging, request/response body capture, slow query detection, and query explain plan injection for local development.

Steps: Logging (debug) → Slow Query Detector → Explain Injector | Body Logger ⚖️ Compliance & Governance GDPR-ready: PII detection, data masking, audit trails, and right-to-erasure hooks for regulated environments.

Steps: Auth → PII Detector → Data Masker → Audit Logger | Compliance Report

Using a Template

Click "Load Template" in the builder toolbar, select a template from the gallery, and click "Apply". The template's steps are loaded onto the canvas with default configurations. You can then modify any step, reorder them, or add additional steps from the catalog.

Loading a template replaces your current canvas. If you have unsaved changes, the builder prompts you to save or discard them first.

Import / Export

Pipeline configurations can be moved between environments, shared with teammates, or version-controlled in Git using the import/export feature.

Export

Click "Export" in the builder toolbar to download the current pipeline configuration:

• JSON — machine-readable format, ideal for CI/CD pipelines and API-driven deployments.
• YAML — human-readable format, ideal for version control and manual review.

## Exported pipeline: production-observability.yaml
## Nexo Pipeline v2
## Exported: 2025-01-14T10:30:00Z
## Proxy: my-app-proxy
## Tier: Pro

pipeline:
  version: 2
  name: production-observability
  description: Full observability stack for production workloads
  request:
    - step: auth
      config:
        provider: jwt
        jwks_url: "https://auth.example.com/.well-known/jwks.json"
    - step: logging
      config:
        level: info
        format: json
    - step: tracing
      config:
        exporter: otlp
        endpoint: "https://otel.example.com:4317"
  response:
    - step: metrics
      config:
        export_to: prometheus
    - step: compression
      config:
        algorithm: zstd

Import

Import a pipeline from a local file or a remote URL:

• From file — drag a JSON or YAML file onto the builder canvas, or use the "Import" button to open a file picker.
• From URL — paste a URL pointing to a raw JSON or YAML file (e.g., a GitHub raw link) and the builder fetches and loads it.

Cross-Environment Copy

A common workflow is to promote a pipeline across environments:

1. Design and test the pipeline in Development.
2. Export the config and import it into Staging.
3. Run integration tests and validate with production-like traffic.
4. Export from staging and import into Production.
5. Deploy using the Canary strategy for a safe rollout.

Environment-specific values (e.g., secret references, endpoints) can use Nexo's variable substitution syntax: ${env.AUTH_JWKS_URL}. This way, the same exported file works across environments without manual editing.

## Using environment variables in pipeline config
pipeline:
  request:
    - step: auth
      config:
        provider: jwt
        jwks_url: "${env.AUTH_JWKS_URL}"
    - step: logging
      config:
        level: "${env.LOG_LEVEL}"
        output: "${env.LOG_OUTPUT}"
    - step: rate-limiter
      config:
        max_requests: "${env.RATE_LIMIT_MAX}"
        window_seconds: 60

Keyboard Shortcuts

The pipeline builder supports keyboard shortcuts for common actions. These work when the canvas or a step is focused (not when editing a text field in the config panel).

Shortcut	Action	Description
Ctrl+S	Save draft	Save the current pipeline as a draft without deploying.
Ctrl+D	Deploy	Open the deployment modal for the current pipeline.
Ctrl+Z	Undo	Undo the last canvas action (add, remove, move, or configure step).
Ctrl+Shift+Z	Redo	Redo a previously undone canvas action.
Del	Remove selected step	Delete the currently selected step from the canvas.
Ctrl+J	Toggle JSON view	Show or hide the JSON/YAML preview panel.

On macOS, use ⌘ (Command) instead of Ctrl. The builder automatically detects your operating system and shows the appropriate modifier key in tooltips and the shortcuts overlay.

Press ? at any time to open the full keyboard shortcuts overlay.

Tips & Best Practices

• Start with a template — templates encode best practices and save you from common ordering mistakes.
• Auth first, compression last — as a general rule, authentication should be the first request step and compression should be the last response step.
• Use canary for production — always deploy to production using the canary strategy to catch regressions early.
• Version control your configs — export your pipelines and commit them to Git alongside your application code.
• Use environment variables — avoid hardcoding URLs and secrets. Use Nexo's ${env.VAR} syntax for portability.
• Check the cost bar — the real-time cost estimate prevents billing surprises when deploying.
• Review the diff — before deploying changes to an existing pipeline, always review the diff view in the preview panel.

Related Documentation

• Pipeline Concepts — Learn how pipelines work under the hood: step lifecycle, execution model, and error handling.
• Step Reference — Detailed documentation for every built-in step, including all config options and examples.
• Marketplace — Browse and install third-party pipeline steps from the Nexo plugin marketplace.
• Canary Deployments — Deep dive into canary deployment strategies, rollback automation, and traffic shifting.
• Secrets Management — How Nexo stores and injects secrets into pipeline step configurations securely.
• Billing & Tiers — Understand pricing tiers, per-step costs, and how marketplace plugins affect your monthly bill.