CameleeringCameleering / Ledger
Cameleering · the ledger
The ledger: every claim, its baseline and its method
One row per measurement. A project gets numbers here only when it has been measured against a named baseline on the same hardware, with a script anyone can rerun. Losses are listed with the wins.
No claim without a number.
Projects
| project | what it is | status | compared with | last measured |
|---|---|---|---|---|
| Cameleer | integration engine in the style of Apache Camel, one std-only native binary | Cameleered | Apache Camel 4.23.0 on Java 21; Apache Camel 4.22.1 on Quarkus 3.40.0 native | 2026-11-06 |
| simoom | embeddable HTTP/1.1, HTTP/2 and HTTP/3 server engine, std-only Rust | Cameleered | Cameleer's previous HTTP server (cameleer-http); 20 HTTP servers in Rust, C, Go, Java, .NET and JavaScript, among them hyper 1.12.0, actix-web 4.18.2, axum 0.8.9 with rustls, may-minihttp 0.1.11, nginx 1.24.0 and H2O | 2026-11-12 |
| karez | database connectivity shaped like JDBC, ODBC and ADBC, with Arrow results | Cameleering | targets only | — |
| ksar | cryptographic primitives and PKI | planned | — | — |
Cameleer, measured 2026-11-06
Cameleer 0.4.0 against Apache Camel 4.23.0 on Java 21, the same routes on identical DigitalOcean c-4 machines (4 dedicated vCPUs each); each figure is the mean of two runs on two servers. Ratios are rounded down, Cameleer's times and memory up. The start-up ratio is computed from the unrounded means, Apache Camel on the JVM 1,106 ms and Cameleer 2.45 ms, which is 451x and is shown as 450x; the table shows the times rounded (1.1 s and 3 ms), so dividing the table's own figures gives less (summary.json).
| metric | Apache Camel (JVM) | Cameleer | result |
|---|---|---|---|
| time to a started context | 1.1 s | 3 ms | 450x faster |
| memory at startup | 114 MB | 8 MB | 14x less |
| peak memory, 11 scenarios | 214–440 MB | 7–67 MB | 4.8–54x less |
| throughput, 11 scenarios | — | — | 1.7–8.7x |
| p99 latency, REST API with JSON | 6.3 ms | 2.6 ms | 2.4x lower |
| p99 latency, HTTP into Kafka | 7.8 ms | 8.6 ms | 10% higher (loss) |
Against the fastest-starting Camel: Quarkus native
| metric | Apache Camel (Quarkus native) | Cameleer | result |
|---|---|---|---|
| time to a started context | 30 ms | 3 ms | 10x faster |
| memory at startup | 69 MB | 8 MB | 8.6x less |
| throughput, 11 scenarios | — | — | 1.9–9.9x |
Known losses and limits
- p99 latency for HTTP into Kafka is higher than Camel on the JVM (8.6 ms against 7.8 ms) while serving 1.7 times the requests: Cameleer batches concurrent sends.
- Load model: the comparison with Apache Camel uses a closed-loop driver, the same on both sides: N workers each send the next request when the last is answered. Throughput is what each side sustains at saturation, and p99 is the latency at that saturation, timed from when each request was sent, so it is not corrected for coordinated omission. Read the p99 rows as relative (BENCHMARKS).
- Plain HTTP is the smallest throughput gain: a fixed-reply route reaches 1.7x Camel's throughput but 68% of a do-nothing server on the same machines (the HTTP ceiling). simoom is the project meant to close that gap.
Method, every scenario and the raw data: BENCHMARKS and how we tested; raw results in benchmarks/results/2026-11-06-linux, generated by scripts/cloud-bench/summarize.py.
simoom, measured 2026-11-11 and 2026-11-12
simoom 0.1.0, the HTTP engine that replaces Cameleer's thread-per-connection server. Two comparisons: against that original server on one machine, and against 20 other HTTP servers on the same DigitalOcean c-4 droplets (4 dedicated vCPUs), load from two separate nodes, open loop. Throughput is the highest rate whose p99 stays at or under 1 ms.
Against the server it replaces: cameleer-http
| metric | cameleer-http | simoom | result |
|---|---|---|---|
| throughput, 64 connections, server on 4 CPUs (closed loop) | 172k–173k req/s | 293k–311k req/s | 1.7–1.8x |
| p99 latency at that load | 0.87–0.91 ms | 0.32–0.44 ms | 2–2.7x lower |
| server CPU per request | about 23 µs | about 13 µs | 43% less |
| memory with 1,000 idle connections | 7.7 MB | 2.7 MB | 65% less |
| cold start to first response, plain / TLS | 0.77 / 1.37 ms | 0.73 / 0.87 ms | 5% / 37% faster |
| idle memory (PSS), plain | 1,017 KB | 1,018 KB | level |
| idle memory (PSS) with TLS | 1,317 KB | 1,360 KB | 3.3% higher (loss) |
Workstation, WSL2 on 12 CPUs, the same 13-byte reply on both sides: load figures 2026-11-09, start-up and memory 2026-11-11 (10 alternating rounds), TLS memory after the K11 fix of 2026-11-12.
Against 20 other HTTP servers, same droplets: campaign 4, 2026-11-12
hyper 1.12.0, axum 0.8.9, axum 0.8.9 + rustls 0.23.46 (aws-lc-rs), actix-web 4.18.2, ntex 2.18.0, xitca-web 0.7.1, h3 0.0.8 + quinn 0.11.12, may-minihttp 0.1.11, nginx 1.24.0, H2O 2.3.0-DEV, HAProxy 2.8.30, Envoy, Caddy 2.11.9, cameleer-http 0.5.1, Go net/http and fasthttp (go1.27.3), Netty 4.2.20, Vert.x 5.2.3, Kestrel (.NET 10.0.403) and uWebSockets.js 20.71.0, against simoom at commit a81c368 (the benchmark server with every feature). Three lanes of DigitalOcean c-4 droplets in blr1. simoom ran in every lane and every round, before that lane's other servers, and the server droplets rotated between rounds. Each figure is the same-droplet run whose ratio to simoom is the median. "Place" counts the servers more than 3% ahead of simoom.
| metric | best other on the droplet | simoom | result |
|---|---|---|---|
| plaintext GET, 256 connections (3 rounds) | may-minihttp 154,590 req/s | 145,171 req/s | 6% behind, 2nd of 19 (loss) |
| plaintext GET, 1,024 connections (1 round) | may-minihttp 155,048 req/s | 133,243 req/s | 16% behind, 2nd of 19 (loss) |
| plaintext GET, 64 connections (1 round; simoom stalled in its run) | Vert.x 104,411 req/s | 69,415 req/s | Vert.x 50% ahead, 6th of 19 (loss) |
| TLS 1.3 full handshakes per second (3 rounds) | axum + rustls 11,633 | 9,396 | 24% behind, 2nd of 8; 1.39x H2O (loss) |
| TLS 1.3 resumed handshakes per second (3 rounds) | axum + rustls 13,537 | 12,388 | 9% behind, 2nd of 8; 1.56x H2O (loss) |
| plaintext GET over TLS, 256 connections (3 rounds) | axum + rustls 103,343 req/s | 134,004 req/s | +30%, 1st of 8 |
| HTTP/3 GET, 64 connections (3 rounds) | H2O 74,961 req/s | 52,259 req/s | H2O 43% ahead, 2nd of 4; 1.65x quinn + h3 (loss) |
| CPU per request at 50k req/s (3 rounds) | may-minihttp 28.6 µs | 24.3 µs | 15% less, 1st of 19 |
| memory per idle connection, 100k open | nginx 528 B | 532 B | level, joint 1st of 15 |
| memory per idle connection, 10k open | nginx 538 B | 575 B | 7% more, 2nd of 19 (loss) |
| answered within 1 ms at 195k req/s offered (overload) | actix-web 59,015 req/s | 2,743 req/s | 22x fewer, 10th of 19 (loss) |
| all answers at 195k req/s offered | uWebSockets.js 159,633 req/s | 150,344 req/s | 6% fewer; most servers 150k–160k |
| p99 of real requests under 10k slowloris + 1k slow-read connections | Go net/http 445 µs | 791 µs | 78% higher, 6th of 19 (loss) |
| start-up to first answer, default build | hyper 1.48 ms | 0.94 ms | 57% faster, 1st of 21 |
| stripped executable, default build | may-minihttp 515 KB | 735 KB | 43% larger, 2nd of 17 (loss) |
| stripped HTTP/1.1-only executable | may-minihttp 527,424 B | 550,224 B | 4% larger (loss) |
Known losses and limits
- Load model: simoom’s request-rate and latency tests are open-loop (a constant rate, each request timed from when it was due) and its overload test offers a fixed absolute rate. Its TLS handshake, HTTP/3 and bulk-download tests are closed-loop, so those figures are throughput at saturation.
- Under overload simoom answers about as many requests in total as the others (150.3k, against 150k–160k for most servers), but few of them within 1 ms: actix-web answered 59.0k in time and nginx 29k.
- A new connection per request: 5th of 19. xitca-web is 29% ahead, cameleer-http 9%, actix-web 5% and may-minihttp 4%.
- The 64-connection plaintext row is a single round in which simoom stalled, so it is not a fair reading of simoom's throughput at that load.
- The degraded-network result from campaign 3 (2026-11-11: may-minihttp kept 114.7k req/s while simoom and the others fell to 17k–62k) was not retested.
- Larger machines and multi-queue NICs are not measured yet.
Superseded: the 2026-11-12 epoll round and campaign 3 (2026-11-11)
These runs had fewer servers, and droplets of the same type differed by up to 7x between runs. Campaign 4 reran every row with simoom on each server's own droplet.
| metric | best other on the droplet | simoom | result |
|---|---|---|---|
| plaintext GET, 64 / 256 / 1024 connections (median of 3 rounds, 2026-11-12) | may-minihttp 158,598 / 150,673 / 120,610 req/s | 170,648 / 172,638 / 148,429 req/s | +8% / +15% / +23% |
| TLS 1.3 full / resumed handshakes per second | h2o 7,711 / 7,758 | 10,083 / 11,213 | +31% / +45% |
| plaintext GET over TLS, 256 connections | h2o 97,219 req/s | 160,441 req/s | 1.65x |
| HTTP/3 GET | h2o 94,507 req/s | 106,947 req/s | +13% |
| CPU per request at 50k req/s | hyper 28.10 µs | 25.05 µs | 11% less |
| memory per idle connection, 100k open | nginx 532 B | 520–533 B | level |
| answered within 1 ms at 1.17x the NIC ceiling (overload) | nginx 52.4k req/s | 6.4k req/s | loss |
| p99 for legitimate requests during a slowloris attack | haproxy 724 µs | 784 µs | 8% higher (loss) |
| stripped HTTP/1.1-only executable | may-minihttp 515 KB | 536 KB | 4% larger (loss) |
The full crossing report: simoom benchmarks. Method, raw data and analyses: bench/results/2026-11-12-campaign4 (the campaign 4 table, every figure from c4.json at simoom commit ef33bb9); superseded: bench/results/2026-11-12-epoll (plaintext, handshakes), bench/results/2026-11-11-campaign3 (the other rows) and docs/STATUS.md (against cameleer-http); the scripts are in bench/cloud. Source: github.com/Cameleering/simoom. Released as 0.1.0 on crates.io (tag v0.1.0).
How the ledger works
What a row needs
- A named baseline with its exact version: the software people run today.
- Both sides on the same hardware, with the machine type written down.
- A script anyone can rerun, and the raw results.
- Both ends of every range: "4.8–54x", never only "54x".
- The losses, in the same table as the wins.
How rows change
- Cameleered rows carry numbers: measured against a named current baseline on the same hardware, with a rerunnable script, losses disclosed, within 12 months (the labels). Cameleering rows are in progress and show targets only; planned rows are not started. Neither carries numbers.
- A target is shown hollow, like 10x, and never in the result column.
- A result older than 12 months is marked stale until it is rerun.
- Rows are never deleted. A newer run supersedes an old one, which stays visible, struck through.
Measured your own software the same way? The method has the report format.