Every team running self-hosted LLM inference on Kubernetes has stared at the same graph: a replica comes up, and nothing useful happens for the next ninety seconds. The image pulls, the weights come down from object storage, the framework initialises, CUDA graphs compile, the first few requests get sacrificed to warmup. So you stop scaling down. You pin a floor of always-on H100s and pay for idle silicon because the alternative is a traffic spike that lands on a Pod still loading a 140 GB safetensors file.
Google's answer went GA on 21 September 2026: GKE Pod snapshots, which checkpoint a running Pod's full state, GPU memory included, to Cloud Storage and restore it into a fresh Pod. This piece covers what the docs actually say the feature does, what it costs you in constraints, and when the trade is worth making.
Where the ninety seconds actually goes
Break down a typical vLLM or TGI cold start and the time spreads across four phases. Image pull is the smallest problem and the one everyone attacks first, because it is the most visible. Weights download dominates: tens or hundreds of gigabytes from object storage into node-local disk, then host memory, then VRAM. Framework initialisation adds tens of seconds of Python import, tokenizer construction and parallelism setup. Then compile and warmup, where torch.compile or CUDA graph capture burns GPU time before the first token ships.
The usual mitigations each shave one phase. Image streaming helps the pull; a local SSD or read-through weights cache helps the download. Neither touches framework init or compile warmup, which is why teams that have already done the storage work still see tens of seconds of dead time.
What GKE Pod snapshots actually capture
The feature captures the running state of a Pod: memory, threads, file descriptors, CPU registers, the container root filesystem, emptyDir and tmpfs mounts, and loopback, listening and Unix-domain sockets. For GPU workloads it captures device memory too, using NVIDIA's cuda-checkpoint tooling. It does not capture persistent volumes, external network connections (those are closed on restore), or user-configured network rules such as iptables entries.
The mechanism is gVisor. Pods must run under GKE Sandbox, which is what makes the userspace kernel state capturable at all. That is the most consequential detail in the concepts documentation and it is easy to miss in the announcement. If your inference Pods are not already gVisor-compatible, that is your first migration, not snapshots.
Restore is not a bulk memory copy before start. The gVisor kernel loads first, typically in seconds, then the application resumes while memory streams in the background, with page faults on unloaded regions triggering prioritised fetches. Three things change underneath the restored process: a new IP, a new hostname, and a wall clock that jumps to now. Anything that cached its own identity, held a lease, or assumed sane wall time has to re-initialise.
Snapshot-to-Pod matching uses what the docs call a distilled spec hash, comparing image, command, args, volumeMounts and security context. The restoring Pod must also match the original's machine series and CPU architecture. A lighter rootfs-only scope checkpoints just the container root filesystem and works across machine families, but gives up the memory and GPU state that make the feature interesting for inference.
Using it
Cluster-level enablement is a flag. On Autopilot, gcloud container clusters create-auto --enable-pod-snapshots. On Standard you also need Workload Identity (--workload-pool and --workload-metadata=GKE_METADATA) and at least one node pool created with --sandbox type=gvisor. Minimum version is GKE 1.35.3-gke.1234000; the multi-tenant short-lived token path needs 1.35.3-gke.1737000 or later.
The bucket setup is prescriptive. Google's prep guide asks for hierarchical namespace enabled (read and write QPS), soft delete disabled (parallel composite uploads create temporary objects you do not want retained and billed), and colocation with the cluster region.
From there it is three custom resources in the podsnapshot.gke.io/v1 API group. A PodSnapshotStorageConfig points at the bucket:
apiVersion: podsnapshot.gke.io/v1
kind: PodSnapshotStorageConfig
metadata:
name: inference-snapshots
spec:
snapshotStorageConfig:
gcs:
bucket: my-snapshot-bucket
path: snapshots/
# federatedP4SA avoids per-namespace KSA-to-IAM bindings
tokenSource: podKSA
A PodSnapshotPolicy selects Pods by label, references that storage config, sets the trigger type (manual or workload) and carries retention: a lastAccessTimeout for age-out, and snapshotGroupingRules with maxSnapshotCountPerGroup to cap how many snapshots survive per label group. Triggering is either a PodSnapshotManualTrigger naming a targetPod, or the workload signal, where the application itself writes echo 1 > /proc/gvisor/checkpoint once it has finished loading and warming. For an inference server, the workload signal is the right one: only the process knows when CUDA graphs are captured.
Restoring a specific snapshot rather than the latest is an annotation on the Pod spec, podsnapshot.gke.io/ps-name. Day-to-day inspection is ordinary kubectl against podsnapshots.podsnapshot.gke.io.
One sizing trap from the trigger guide: a 100 GB model produces roughly a 200 GB snapshot, because you are storing both the weight files on the container filesystem and the same weights again as GPU state. Google's advice is to delete the on-disk weights before triggering the snapshot. That is a real change to your container's startup script, not a config toggle.
Hardware limits
The GPU support matrix is narrower than the announcement implies. Snapshots of GPU state are limited to g2-standard (L4), a2-highgpu and a2-ultragpu (A100), and a3-highgpu (H100). Multi-GPU Pods are supported only on L4; on A100 and H100 you are restricted to single-GPU Pods, which rules out tensor-parallel 70B deployments that span four or eight devices on one node. Multi-Instance GPU is unsupported. TPUs are unsupported. E2 machine types cannot use the default whole-pod scope. The Cloud Storage FUSE CSI driver sidecar is incompatible.
The scheduling side has moved separately: our write-up of DRA extended resources reaching GA in Kubernetes 1.37 covers allocating accelerators without a device plugin, a different layer of the same problem.
How this sits with autoscaling
The value is at the bottom of the scaling curve, not the top. If your HPA or KEDA ScaledObject has minReplicas: 3 because two of those are insurance against cold starts, snapshots are the argument for one, or for scale-to-zero on bursty internal workloads. The restore still needs a node with a matching machine series, so autoscaler and node auto-provisioning time does not disappear; you remove the model-loading phase, not the VM boot. Node auto-provisioning has to be pinned to the right machine families or restores fail the hardware match.
Note that snapshots live in a bucket colocated with the cluster region, so your fast restore path has a regional dependency. That is worth folding into the same thinking as what a regional failure really does to multi-AZ designs: a cross-region failover cluster does not inherit your snapshots for free.
Comparing the options
Approach | Startup improvement | Ongoing cost | Main trade-off |
|---|---|---|---|
Larger always-on replica floor | None; you pay to avoid the problem | Idle GPU hours, continuously | Most expensive option per unit of protection |
Image streaming and local SSD weight cache | Removes pull and download phases only | Local SSD on every node | Framework init and CUDA warmup remain |
Model weight caching service (read-through) | Removes repeated download | Cache tier plus egress | Cold cache on new node pools; still no warmup saving |
GKE Pod snapshots (whole-pod) | Removes all four phases | Object storage for ~2x model size per snapshot | gVisor requirement, narrow GPU matrix, single-GPU on A100/H100 |
GKE Pod snapshots (rootfs-only) | Removes pull and download | Smaller snapshots | No memory or GPU state; warmup still runs |
The caveats that will come up in review
Storage cost is not nominal: roughly twice your model size per snapshot, multiplied by however many variants and revisions you keep. Google does not publish a separate feature charge, so it lands as Cloud Storage capacity and operations. Set lastAccessTimeout and maxSnapshotCountPerGroup on day one rather than finding the growth curve in month three.
A memory image contains whatever was in memory. If your server pulled a service account token, an API key or a decrypted license at startup, that material now sits in an object in a bucket and gets restored into every future replica. Short-lived tokens stop being short-lived when you freeze them. Treat the bucket as a secrets store for access control, use the federatedP4SA token source to avoid broad KSA bindings, and move secret retrieval after the checkpoint point in your startup sequence where you can.
Staleness is a deployment-process problem. The distilled hash pins a snapshot to a specific image and spec, which prevents the worst case, but nothing stops you restoring last quarter's weights because they happened to match. Snapshot identity belongs in your release pipeline, not beside it.
This is GKE-only, built on gVisor, with GKE-specific CRDs. Upstream Kubernetes has had container checkpointing as an alpha kubelet feature for some time, but it is a different mechanism with a different scope, so the migration path off this is rebuilding your loading strategy, not swapping a CRD. Our Kubernetes architecture walkthrough covers the control plane and node components snapshots hook into; more accelerator platform patterns are in our AI infrastructure section.
The decision rule
Adopt Pod snapshots if three things are true at once: your inference Pods fit in a single L4, A100 or H100 GPU; your scaling pattern is genuinely bursty, so idle GPU hours are a line item you can point at; and you can run the workload under gVisor without a performance regression you care about. Test that last one before anything else, because it is the condition most likely to fail.
Skip it if you run multi-GPU tensor parallelism on A100 or H100, if you are on TPUs, or if your load is steady enough that a fixed replica floor is close to fully utilised anyway. In that last case you are paying storage to solve a problem you do not have.
Google reports up to an 89% reduction in startup latency for llama3-70b, roughly 37 seconds for 70B models and 15 seconds for 8B, and cites Retake by Codeway moving from about a minute to about eight seconds on A3 H100 instances. Those are vendor figures on vendor workloads. Measure your own: snapshot one replica, restore it fifty times, and watch p99 time-to-first-token rather than time-to-ready. With page-fault streaming, "running" and "serving at full speed" are not the same moment.
