Built for Speed
Measured 2026-10-06 on Ferrum Edge v0.9.13+ (commit e5c761627, 11 commits after the v0.9.13 tag) across 10 protocols: client → gateway → backend, each with a direct baseline over the same client protocol. Every protocol figure is the median of 3 runs (the 30,000-proxy scale test is a single run), produced by scripts in the repository that you can run yourself.
How We Measure
Hardware & Build
Apple M4 (10 cores, 16 GB), macOS 26.6.1 (arm64). Gateway built with cargo build --release (fat LTO, jemalloc). Load generator, gateway, and backend all share this one machine over loopback, with ordinary desktop apps running.
Load Generator
proto_bench: closed-loop, 200 concurrent connections, a warmup request per connection, then a 15-second measured window with HDR-histogram latency. Every response is checked byte for byte. 3 runs per row, alternating gateway-first and direct-first order.
Backend & Baseline
proto_backend echoes each request body back. The direct leg runs the same client protocol straight to the backend (HTTPS for HTTP/1.1 + TLS, TLS for TCP + TLS, DTLS for UDP + DTLS), so the difference is the gateway hop alone.
Protocol Performance Comparison
64-byte echo, 200 concurrent connections, 15-second measured window, median of 3 runs. Across every suite, 0 errors were recorded and the highest run-to-run variation in gateway RPS was 1.1%.
| Protocol | Gateway RPS | Direct RPS | RPS overhead | Gateway CPU / req | Gateway p50 / p99 | Added p50 (light load) | Notes |
|---|---|---|---|---|---|---|---|
| HTTP/1.1 | 104,182 | 211,967 | 51% | 54 µs | 1.84 ms / 3.85 ms | 30 µs | Pooled keep-alive backend connections |
| HTTP/1.1 + TLS | 102,285 | 211,964 | 52% | 55 µs | 1.87 ms / 3.98 ms | 32 µs | TLS ends at the gateway; direct leg is HTTPS |
| HTTP/2 (TLS) | 46,987 | 274,902 | 83% | 96 µs | 4.30 ms / 5.66 ms | 38 µs | TLS + h2 on both hops |
| HTTP/3 (QUIC) | 59,801 | 105,272 | 43% | 80 µs | 3.33 ms / 4.99 ms | 47 µs | QUIC on both hops |
| WebSocket | 108,566 | 211,876 | 49% | 36 µs | 1.82 ms / 2.58 ms | 15 µs | Frames tunnelled after upgrade |
| gRPC (h2c) | 79,694 | 213,170 | 62% | 49 µs | 2.36 ms / 5.54 ms | 39 µs | Unary echo over h2c |
| TCP Proxy | 108,970 | 210,983 | 48% | 35 µs | 1.82 ms / 2.45 ms | 13 µs | Bidirectional byte copy |
| TCP + TLS | 108,169 | 211,378 | 49% | 36 µs | 1.82 ms / 2.57 ms | 15 µs | TLS ends at the gateway |
| UDP | 81,918 | 261,483 | 69% | 59 µs | 2.50 ms / 2.89 ms | 14 µs | Per-datagram session forwarding |
| UDP + DTLS | 75,031 | 97,264 | 23% | 78 µs | 2.64 ms / 3.69 ms | 15 µs | DTLS ends at the gateway |
Gateway CPU / req: gateway process CPU time ÷ requests it served during the leg. Added p50 (light load): gateway p50 minus direct p50 with a single connection, which is the time the extra hop adds when nothing is queued. Latency at 200 connections is mostly queueing, so compare it between rows rather than reading it as hop cost.
10 KB payloads
Same setup with a 10 KB request echoed back as a 10 KB response (UDP rows use 2 KB datagrams to stay under the loopback MTU). Body copying, encryption, and framing grow with the payload, so this is closer to real API traffic than the 64-byte run.
| Protocol | Gateway RPS | Direct RPS | RPS overhead | Gateway CPU / req | Gateway p50 / p99 | Notes |
|---|---|---|---|---|---|---|
| HTTP/1.1 | 96,609 | 210,533 | 54% | 59 µs | 1.99 ms / 4.14 ms | Pooled keep-alive backend connections |
| HTTP/1.1 + TLS | 84,744 | 178,335 | 52% | 67 µs | 2.25 ms / 4.97 ms | TLS ends at the gateway; direct leg is HTTPS |
| HTTP/2 (TLS) | 36,313 | 169,660 | 79% | 114 µs | 5.56 ms / 7.81 ms | TLS + h2 on both hops |
| HTTP/3 (QUIC) | 7,379 | 11,624 | 37% | 641 µs | 26.09 ms / 55.42 ms | QUIC on both hops |
| WebSocket | 104,860 | 200,604 | 48% | 35 µs | 1.88 ms / 2.84 ms | Frames tunnelled after upgrade |
| gRPC (h2c) | 57,475 | 112,973 | 49% | 66 µs | 3.23 ms / 8.57 ms | Unary echo over h2c |
| TCP Proxy | 99,220 | 193,977 | 49% | 28 µs | 2.01 ms / 2.28 ms | Bidirectional byte copy |
| TCP + TLS | 93,043 | 159,829 | 42% | 37 µs | 2.14 ms / 2.52 ms | TLS ends at the gateway |
| UDP | 81,106 | 251,031 | 68% | 59 µs | 2.52 ms / 2.93 ms | Per-datagram session forwarding (2 KB datagrams) |
| UDP + DTLS | 74,926 | 95,677 | 22% | 77 µs | 2.64 ms / 3.62 ms | DTLS ends at the gateway (2 KB datagrams) |
Data Structure Complexity
Hot-path lookups are hash-indexed: O(1) on route-cache hits and O(path depth) on misses, with no scan over the route list as configurations grow. Only the ArcSwap snapshot reads are lock-free; DashMap reads take a per-shard read lock.
| Component | Data Structure | Lookup | Update | Concurrency |
|---|---|---|---|---|
| Global Config | ArcSwap<Config> |
Atomic load | Atomic swap | Lock-free reads, single writer |
| Route Cache (hit) | DashMap<(Host, Path), Match> |
O(1) average | Insert on miss, shard-locked | Per-shard read locks; bounded, negative results cached |
| Route Table (miss) | ArcSwap snapshot: HashMap indexes + RegexSet |
O(path depth) prefix; O(path length) regex | Rebuilt atomically on config change | Lock-free snapshot read; wildcard-host patterns scanned linearly |
| Plugin Registry | DashMap<Name, Plugin> |
O(1) | O(1) shard-locked | Concurrent reads |
| Consumer Store | DashMap<Key, Consumer> |
O(1) | O(1) shard-locked | Concurrent reads, shard-locked writes |
| Rate Limit State | DashMap<Key, Bucket> |
O(1) | Atomic CAS | Lock-free per-key updates |
| Connection Pool | Per-host Vec + Mutex |
O(1) + lock | O(1) + lock | Short-duration lock on pool access only |
| Health State | AtomicBool per upstream |
Atomic load | Atomic store | True lock-free |
What Happens on Every Request
Config Load — O(1), lock-free
Single atomic pointer load from ArcSwap. Returns a reference-counted Arc to the current config. No mutex, no blocking.
Route Lookup — O(1) on a cache hit, O(path depth) on a miss
Repeated (host, path) pairs are served from a bounded DashMap cache. On a miss the ArcSwap route table is searched by host tier (exact, wildcard, catch-all), then by exact path, an indexed longest-prefix walk over the path's segments, and finally regex routes in one RegexSet pass. Cost is independent of route count except for the wildcard-host scan.
Plugin Execution — O(P)
Linear scan over sorted plugin list (P = number of enabled plugins, typically 1-5). Each plugin executes its hook in priority order. Most requests have <5 plugins.
Upstream Selection — O(1) to O(log N)
Load balancing algorithm selects upstream. Round robin is O(1) with an atomic counter. Least connections requires O(N) scan but N is typically small (<20).
Connection Pool — O(1) + brief lock
Per-host pool lookup. Brief mutex to pop a connection from the pool (microseconds). HTTP/2 pools are almost always lock-free (multiplex on existing connection).
Proxy + Response — network-bound
Forward request to upstream, await response. This dominates total latency. All Ferrum Edge overhead is in steps 1-5 and is measured in microseconds.
Run the Same Benchmarks
Don't take our word for it. The harness that produced this page is in the repository, pinned below to the exact commit (kept reachable by the tag bench/2026-10-06-apple-m4), along with the raw results. It needs a Rust toolchain, protoc, python3, curl, and lsof. The harness is source available under the repository's PolyForm Noncommercial or a commercial license.
git clone https://github.com/ferrum-edge/ferrum-edge.git
cd ferrum-edge
git fetch origin tag bench/2026-10-06-apple-m4
git checkout --detach e8d16e8e2c6710cf1b86dbcfc792719e51c3c6c3
# All 10 protocols: 64 B and 10 KB throughput, light-load latency,
# 3 runs each with alternating leg order (about 55 minutes on an M4)
cd tests/performance/multi_protocol
./run_published_benchmark.sh
# Faster check: two protocols, one payload, one run
./run_published_benchmark.sh --protocols "http1 grpc" --payload-sizes 64 --runs 1
# Results land in results/<timestamp>/summary.md (plus summary.json,
# samples.json, and manifest.json)
# 30,000-proxy scale test (SQLite, about 25 minutes plus build time)
cd ../../..
cargo build --release --bin ferrum-edge
FERRUM_SCALE_RESULTS_JSON=scale.json \
cargo test --profile ci-release --test functional_tests -- --ignored --nocapture \
--exact functional::functional_scale_perf_test::test_scale_perf_30k_proxiesFlags column in summary.md before you publish a row.
Artifacts from this run
| File | Contents |
|---|---|
summary.md | Readable tables with every column, flags, and spread |
summary.json | Medians, min/max, CV, and per-run values for every row |
samples.json | Every raw proto_bench report and gateway CPU sample |
manifest.json | Commit, version, toolchain, host, load average, and arguments |
scale-sqlite.json | 30,000-proxy scale run, every batch |
Also downloadable from this site: summary.json · scale-sqlite.json
Harness files
| Path | Purpose |
|---|---|
tests/performance/multi_protocol/run_published_benchmark.sh | One command that produced every protocol number on this page |
tests/performance/multi_protocol/run_protocol_test.sh | Single-protocol runner: starts the backend and gateway, then measures both legs |
tests/performance/multi_protocol/summarize_published_benchmark.py | Turns raw runs into medians, spread, CPU per request, and validity flags |
tests/performance/multi_protocol/configs | Gateway configs for each protocol |
tests/functional/functional_scale_perf_test.rs | 30,000-proxy scale test |
tests/functional/SCALE_PERF_TEST.md | Scale test guide |
Performance at 30,000 Proxies
Multi-tenant simulation: 30,000 proxies, each secured with key_auth + access_control and given its own consumer. Each wave of 3,000 is added to the running gateway (SQLite, no restart) and must go live before the next window is measured.
| Proxies | RPS | Avg (ms) | P50 (ms) | P95 (ms) | P99 (ms) | Max (ms) | CPU / req | % of 3k |
|---|---|---|---|---|---|---|---|---|
| 3,000 | 88,874 | 0.6 | 0.5 | 0.8 | 1.0 | 3.3 | 56.5 µs | 100% |
| 6,000 | 88,355 | 0.6 | 0.5 | 0.8 | 1.0 | 3.2 | 56.3 µs | 99% |
| 9,000 | 88,229 | 0.6 | 0.5 | 0.8 | 1.0 | 3.7 | 56.9 µs | 99% |
| 12,000 | 87,465 | 0.6 | 0.6 | 0.8 | 1.0 | 3.3 | 58.0 µs | 98% |
| 15,000 | 86,972 | 0.6 | 0.6 | 0.9 | 1.0 | 20.8 | 57.5 µs | 98% |
| 18,000 | 87,024 | 0.6 | 0.6 | 0.9 | 1.0 | 4.3 | 58.3 µs | 98% |
| 21,000 | 86,967 | 0.6 | 0.6 | 0.9 | 1.0 | 3.2 | 58.6 µs | 98% |
| 24,000 | 86,762 | 0.6 | 0.6 | 0.9 | 1.1 | 3.2 | 58.5 µs | 98% |
| 27,000 | 86,663 | 0.6 | 0.6 | 0.9 | 1.1 | 13.9 | 57.9 µs | 98% |
| 30,000 | 86,736 | 0.6 | 0.6 | 0.9 | 1.0 | 3.0 | 59.0 µs | 98% |
2026-10-06, same host as above, SQLite. 50 concurrent workers, 5 s discarded warmup, then a 30 s measured window per wave, in a single run; the test binary (load generator + echo backend) is built with the optimized ci-release profile. The load generator shares the machine with the gateway, so watch CPU per request alongside RPS.
View full test details →
RPS as a share of the 3k-proxy run
Why This Matters
Route lookups, plugin resolution, and consumer authentication in Ferrum Edge use pre-computed indexes and hash lookups, so their cost doesn't grow with the number of configured proxies. Going from 3,000 to 30,000 proxies changed throughput by 2.4% in this run, and gateway CPU per request went from 56.5 µs to 59.0 µs.
Each wave of 12,000 resources was written through the admin batch API while the gateway kept running. It went live through an atomic ArcSwap config swap with no restart. This test doesn't send traffic while a wave is being applied, so it shows that config growth is cheap. It doesn't measure latency during a reload.
Benchmark Provenance
Measured 2026-10-06 (UTC) at commit e5c761627 (Ferrum Edge v0.9.13+, 11 commits after the v0.9.13 tag; rustc 1.99.0 (b940084d7 2026-09-28)) on Apple M4 (10 cores, 16 GB), macOS 26.6.1 (arm64), with a load average of 4.94, 9.48, 8.96 at the start. Every figure on this page comes from the published result bundle, which contains the raw samples and the full manifest. These are single-machine loopback results, not a guarantee for other hardware, networks, or configurations.