Cold start is a recovery problem

The question: what is a server worth in its first minutes?

During the Microsoft 365 outage of 12 and 13 November 2026, Microsoft said that servers returned to service "can become unresponsive again once they receive traffic". The headroom essay covers that incident. Microsoft has not published a root cause, and this page does not guess at one. It takes the general lesson only: a server that has just started is not yet a whole server.

So how much of a server is it, and for how long? That number decides how long a fleet runs short after a crash, a deploy or a scale-out. The degraded-fleet test gives the procedure for measuring it. This page explains what to measure and why the usual start-up figure is not enough.

Four clocks, not one

"Start-up time" hides four different intervals. Each starts at process start and ends later than the one before.

Most published start-up numbers, ours included, are the first clock or the second. They are easy to measure on one machine with no load. The fourth clock needs a fleet, a load generator and a target, so it is rarely published. It is also the one that decides recovery.

Why the gap between ready and full speed is the dangerous part

A node between clock 2 and clock 4 passes its health check but cannot yet serve its share. The load balancer sees a healthy node and gives it a full share. Every request misses a cold cache, opens a new database connection or does a full TLS handshake. Latency climbs, the health check fails, and the node leaves. The rest of the fleet carries its load for another cycle, and then the node tries again. That is flapping, and Test 2 of the degraded-fleet test counts it.

Google's SRE book names both halves of this. Processes "are often slower at responding to requests immediately after starting than they will be in steady state", from connection set-up and JIT warm-up; and with an empty cache, "100% of requests are costly". Its advice is to add load to a cluster slowly and to test how the service behaves after a large restart ^[1].

The danger grows when the cold state feeds itself. Bronson and colleagues describe metastable failures: a trigger pushes a system into a bad state that persists after the trigger is removed, held there by a sustaining effect. One of their examples is a look-aside cache: lose a cache with a 90% hit rate and the database sees ten times the queries, answers too slowly for the cache to refill, and stays overloaded ^[2]. A fleet of cold nodes can sit in that state long after the original fault is gone.

The arithmetic of recovery

Time to add capacity = detect + provision + start + warm. Every number in this section is an example, not a measurement.

Take 12 nodes at 70% and lose a third. The load is 8.4 nodes' worth, and 8 nodes remain, so the survivors run at 105%. To get them back under 80% you need 11 nodes, three more. Until those three are at full speed, the fleet is past its knee.

Read it honestly. When provisioning and warm-up dominate, as in the first two rows, cutting start-up from 30 seconds to one saves a tenth of the outage window. For many systems that is the true picture, and the money is better spent on warm pools of spare capacity, faster detection or a cache that survives restarts. Start-up dominates only when the rest is small: restarts in place, crash loops, pods on nodes that are already up, services with little to warm. There, the same cut more than halves the window.

Flapping multiplies everything. If a node fails its health check three times before it holds, the start and warm columns are paid three times while the fleet stays short.

What makes start slow, and the ladder of fixes

Climb it one rung at a time, as on the rewrite ladder. Most teams stop well before the top.

Rung 4 in detail

• GraalVM Native Image compiles Java ahead of time to a binary that starts in milliseconds, under a closed-world assumption: code reached through reflection, JNI and other dynamic features must be declared in metadata ^[4]. Quarkus builds on it and names the costs plainly: native builds take minutes and several gigabytes of build-host memory, where JVM builds take seconds ^[5]. In our own comparison, Apache Camel on Quarkus native started far faster than on the JVM but handled fewer messages per second, since it has no JIT compiler (BENCHMARKS).
• AWS Lambda SnapStart initializes the function when you publish a version, snapshots the memory and disk of the execution environment, and resumes new environments from it. Your code must restore uniqueness (IDs, secrets, random seeds) and re-establish network connections after a resume. It supports Java 11 and later, Python 3.12 and later and .NET 8 and later, and not provisioned concurrency; for Python and .NET there are caching and restore charges ^[6].
• CRaC (Coordinated Restore at Checkpoint) is an OpenJDK project that starts a Java application from an image of a warmed-up running instance. Open files and sockets cannot be saved, so the application must close them before the checkpoint and reopen them after restore ^[7].

Rung 5 in detail

In Kubernetes, a startup probe holds off the liveness and readiness probes until the application has started, and is the recommended tool for slow starters; a failed readiness probe removes the pod from its Services, and a failed liveness probe restarts it ^[3]. A liveness probe that fails under load turns a warming node into a crash loop. The load balancer settings for slow start, and their limits, are listed in Test 2.

The TLS session cache is a case in point: the cost of a TLS handshake shows what an empty one costs when clients reconnect together.

Native or JIT-free does not mean warm. A snapshot or a native binary removes compilation warm-up. Caches, connection pools and TLS sessions still start empty, unless the snapshot carries them and they are still valid on restore.

The money

Since 1 August 2025 AWS Lambda bills the initialization phase of on-demand functions on managed runtimes with ZIP packaging; custom runtimes, container images and provisioned concurrency already paid for it ^[8]. The first two clocks are now on the invoice. The FinOps page has the arithmetic; one line of it, at list price for 512 MB functions: a million cold starts cost $9.17 at 1.1 s and $0.03 at 3 ms. That is arithmetic on our measured start-up times, not a measured bill.

Our own numbers, and what they do not show

These are the only start-up figures we have. Each names its project, its baseline and how it was measured. Every result, good and bad, is in the ledger.

• Cameleer: DigitalOcean c-4 machines, 2026-11-06. The table shows rounded times; the means are 1,106 ms and 2.45 ms. Start-up is the time from spawning the process to its READY line, median of five (BENCHMARKS).
• simoom against 20 servers: campaign 4, DigitalOcean c-4 droplets, 2026-11-12, one round, each server against simoom on the same droplet (simoom benchmarks).
• simoom against cameleer-http: one workstation, WSL2 on 12 CPUs, 2026-11-11, 10 alternating rounds.

The limits, said plainly

• These are the first or second clock. We have not measured time to full speed, or a restart under load, for either project.
• We have not run the full degraded-fleet test on our own projects. Until we do, we cannot say how either behaves in the window this page is about.
• A 3 ms start-up does not by itself make a node safe to give a full share. Caches and pools still fill; the ramp still matters.
• These figures say nothing about Microsoft's systems, or about the outage.

A start-up and recovery checklist

Ten items a team can start next week. Record each result with its build and date, as the method asks.

1. Log a timestamp at process start, at listening and at ready, in every service.
2. Restart one node under steady load in staging and record all four clocks, with and without slow start.
3. Count health-check failures after a restart. More than zero means the node joins too early.
4. Check that readiness means "can serve now" and liveness means "is stuck"; add a startup probe for slow starters.
5. Turn on slow start in the load balancer, and check that your balancing algorithm supports it.
6. List what each start does eagerly: connections, migrations, cache loads, scans. Move what you can off the start path.
7. Measure the cache hit rate in the first minutes after a restart, and decide whether caches should survive one.
8. Write down detect, provision, start and warm for your scale-out path, and find which one dominates.
9. Add a start-up budget per release to CI that fails the build on a regression, as step 6 of the playbook describes.
10. Before trying snapshots or native images, write down what they would cost you in builds, configuration and peak throughput.

Sources

Checked on 2026-11-13. Microsoft's status update is quoted and sourced in the headroom essay.

1. Google, Site Reliability Engineering, chapter 22, "Addressing Cascading Failures": slow start-up and cold caching.
2. Nathan Bronson, Abutalib Aghayev, Aleksey Charapko and Timothy Zhu, "Metastable Failures in Distributed Systems", HotOS '21 (PDF).
3. Kubernetes, liveness, readiness and startup probes.
4. GraalVM, Native Image reference manual.
5. Quarkus, building a native executable.
6. AWS, improving startup performance with Lambda SnapStart.
7. CRaC, Coordinated Restore at Checkpoint documentation.
8. AWS Compute Blog, "AWS Lambda standardizes billing for INIT Phase".
9. Cameleering measurements: the ledger, Cameleer BENCHMARKS and simoom benchmarks.