Performance
See identical-hardware benchmarks for Sluis, LiteLLM, and Bifrost, including gateway overhead, throughput, tail latency, and live-model methodology.
One upstream. Identical hardware. The same load.
Every gateway fronts the same keep-alive mock at the same concurrency, isolating gateway overhead from provider variance. The run exercises the OpenAI protocol path; the Anthropic surface adds one translation step on top.
One shared mock upstream
All four gateways hit the same Node mock: keep-alive on, one fixed 60 ms delay.
Isolated, identical hardware
Each runs in a CPU/memory-pinned container, hit by oha at 256 connections.
Gate-off baseline
Allow-all policy, DLP observe-only, audit content off: Sluis as a plain OpenAI-compatible proxy.
Gate-on: the real product
EU residency, DLP masking, content retention, hash-chained audit: every gate on the hot path.
The numbers, exactly as measured.
Two Sluis columns: gate-off, an apples-to-apples proxy comparison; gate-on, the compliance overhead against our own baseline.
2026-07-04 · oha 1.14 · c=256 · 30 s
| Metric | Sluis gate-off | Sluis gate-on | LiteLLM | Bifrost |
|---|---|---|---|---|
| Throughput (RPS) | 4,046 | 3,678 | 298 | 3,148 |
| p50 latency (ms) | 62 | 63 | 743 | 80 |
| p99 latency (ms) | 77 | 130 | 4,584 | 118 |
| Peak RSS (MiB) | TBD | TBD | TBD | TBD |
| Success rate | 100% | 100% | 100% | 100% |
Latency is end-to-end through the gateway over the shared fixed-latency mock: the differences are the gateways' own overhead.
The gate-on column is compliance overhead versus gate-off, not a head-to-head.
Against a live model (rate-capped at 10 RPS so the provider never throttles), the gate adds nothing at the median: 305 ms gate-on versus 304 ms gate-off. The p99 tails in that lane belong to the provider, not the gateways.
An honest claim, or none at all.
A benchmark you can't trust is worse than none, so we set the rules before any number ships.
- The head-to-head “fastest” claim is gate-off only: Sluis as a plain proxy versus LiteLLM and Bifrost.
- Gate-on is overhead against our own baseline: one millisecond at the median, nine percent throughput, p99 130 ms versus 77 ms.
- The rate-capped live lane adds nothing at the median; its p99 tails are provider variance.
- If gate-off Sluis isn't clearly faster, the headline becomes “compliance at negligible cost”.