Guides / Testing & Validation

Testing & Validation

Ferrum Edge ships its test tooling in the gateway itself: config validation, backend mocking, chaos fault injection, shadow traffic mirroring, and on-demand load testing — plus reproducible benchmark suites in the repository.

Step 1

Validate Configuration Before You Deploy

The validate subcommand checks a config without starting the gateway — perfect as a CI gate.

bash
# Validate a file-mode spec
ferrum-edge validate --spec resources.yaml

# Validate settings + spec together
ferrum-edge validate --settings /etc/ferrum/ferrum.conf --spec /etc/ferrum/resources.yaml

# Use as a CI gate — nonzero exit on any error
ferrum-edge validate --spec resources.yaml || exit 1

Validation catches duplicate listen paths, malformed plugin configs, unknown plugin names, invalid credential shapes, and schema mismatches — the same checks the gateway performs at startup.

Step 2

Smoke-Test a Running Gateway

bash
# Liveness — no auth, always {"status":"ok"} while the process runs
curl http://localhost:9000/live

# Readiness — no auth returns status+ready; full diagnostics require a JWT
curl http://localhost:9000/health

# Authenticated deep health + runtime metrics snapshot
curl -H "Authorization: Bearer $TOKEN" http://localhost:9000/health
curl -H "Authorization: Bearer $TOKEN" http://localhost:9000/metrics/runtime

# Send a request through a configured proxy
curl -i http://localhost:8000/api/v1/hello
# On errors, X-Gateway-Error and X-Gateway-Upstream-Status headers explain why
ℹ️
Debugging a request? Attach the transaction_debugger plugin in development — it prints every plugin decision, timing, and routing outcome per request with sensitive headers redacted.
Step 3

Mock Backends with response_mock

Develop and contract-test against APIs that don't exist yet — rules are scoped to the proxy's listen path, and unmatched requests can pass through to the real backend.

yaml
plugin_configs:
  - id: "orders-mock"
    plugin_name: "response_mock"
    scope: proxy
    enabled: true
    config:
      passthrough_on_no_match: true   # unmatched requests hit the real backend
      rules:
        - method: GET
          path: "/orders/~\\d+"        # regex paths use the ~ prefix
          status: 200
          headers:
            content-type: application/json
          body: '{"id": 1, "status": "shipped"}'
          delay_ms: 150                # simulate realistic latency
        - method: POST
          path: "/orders"
          status: 201
          body: '{"id": 2, "status": "created"}'
Step 4

Chaos-Test Clients with fault_injection

Prove your clients handle failure before production does it for you. Works on HTTP, gRPC, TCP, and UDP.

yaml — HTTP aborts and latency
plugin_configs:
  - id: "chaos-http"
    plugin_name: "fault_injection"
    scope: proxy
    enabled: true
    config:
      abort:
        percentage: 5.0     # 5% of requests
        http_status: 503
      delay:
        percentage: 20.0    # 20% of requests
        duration_ms: 500
yaml — TCP / UDP stream faults
plugin_configs:
  - id: "chaos-stream"
    plugin_name: "fault_injection"
    scope: proxy
    enabled: true
    config:
      stream:
        connect_reject_percentage: 2.0
        connect_delay_ms: 200
      datagram:
        abort_percentage: 1.0
        delay_ms: 50

Per-instance counters keep proxy- and group-scoped experiments independent, so you can chaos-test one route without touching the rest of the gateway.

Step 5

Shadow-Test with request_mirror

Duplicate a sample of live traffic to a new service version. Fire-and-forget — the mirror can be slow or down without affecting a single client response.

yaml
plugin_configs:
  - id: "shadow-v2"
    plugin_name: "request_mirror"
    scope: proxy
    enabled: true
    config:
      mirror_scheme: https
      mirror_host: "orders-v2.internal"
      mirror_port: 443
      sample_percentage: 10.0     # deterministic, evenly spaced sampling
      forward_body: true          # include bodies (bounded budgets apply)

Mirror outcomes flow into transaction logs and Prometheus counters (ferrum_request_mirror_*), so you can compare the shadow service's behavior against production before cutting over. Origin-bound credentials are stripped from mirrored requests by default.

Step 6

On-Demand Load Tests with load_testing

Trigger a load test with a single header — the gateway spawns virtual clients that send requests back through its own proxy listener, exercising the full pipeline: routing, auth, rate limiting, backend dispatch, and logging.

yaml — attach the plugin
plugin_configs:
  - id: "loadtest"
    plugin_name: "load_testing"
    scope: proxy
    enabled: true
    config:
      trigger_key: "${LOADTEST_KEY}"   # required in X-Loadtesting-Key
      virtual_clients: 200
      duration_seconds: 30
      ramp_up_seconds: 5
      # Fan out to remote gateway instances for multi-node tests:
      # gateway_addresses: ["https://gw-2:8443", "https://gw-3:8443"]
bash — trigger it
curl -H "X-Loadtesting-Key: $LOADTEST_KEY" http://localhost:8000/api/v1/hello
# Watch results in your metrics: latency histograms, status windows, pool stats
curl -H "Authorization: Bearer $TOKEN" http://localhost:9000/metrics/runtime
Step 7

Run the Repository Test & Benchmark Suites

Building from source? The repo includes the full test pyramid and the reproducible benchmark harnesses behind our published numbers.

bash
git clone https://github.com/ferrum-edge/ferrum-edge.git
cd ferrum-edge

# Unit + integration + E2E tests (same gate as CI)
cargo test --all-features

# Lint at CI strictness
cargo clippy --all-targets --all-features -- -D warnings
cargo fmt --check

# Multi-protocol performance suite (HTTP/1.1/2/3, gRPC, WS, TCP, UDP)
# See tests/performance/multi_protocol/README.md for options
cd tests/performance && ./run_all.sh

# Docker gateway comparison harness (Ferrum vs Envoy, Kong, Tyk)
cd comparison && ./run_comparison.sh

# Gateway API / Istio conformance matrix
cargo test --test conformance_tests
ℹ️
Functional test guides: the repository documents scenario-based functional testing for auth/ACL, database modes, file mode, and load/stress in docs/functional_testing.md and its companions — useful as templates for testing your own configuration.

Tested and Ready?

Running Modes Guide → See Our Benchmark Results Deploy to Kubernetes