From GPU Waste Finding to Production Change: What Actually Happens in Between

The approval is an intent, not a change

When an engineer clicks approve on a GPU optimization recommendation, they have expressed intent. Nothing in the cluster has changed yet. The pod is still on the wrong tier. The KV cache is still saturated. The OOM kills are still happening.

What approval does is authorize actuation — it says "this recommendation is correct and I am accountable for it." That authorization needs to be captured with an immutable record: who approved, when, under what identity, for which workload. Without that record you have a button, not a workflow.

The change itself requires a separate step: getting the new configuration into the cluster through whatever mechanism that cluster is managed by.

---

Three deployment integration patterns

How a configuration change reaches a Kubernetes cluster depends entirely on how that cluster is managed. There is no universal answer. Optimization tools that assume a single path fail in production environments.

GitOps (ArgoCD / Flux)

The most common pattern in mature AI infrastructure teams. The desired state of every deployment lives in a Git repository. ArgoCD or Flux continuously reconciles the cluster against that repository.

In this model, an optimization recommendation becomes a pull request. Paralleliq opens a PR against the config repository with the specific change: a nodeSelector update, a vLLM argument flag, a replica count. A second engineer reviews the diff and merges it. ArgoCD detects the merge and applies the change to the cluster automatically.

The audit chain has two humans: the operator who approved the recommendation in Paralleliq, and the engineer who reviewed and merged the PR in Git. Both are in the record.

Helm-managed deployments

Helm stores the current release state — values, chart version, computed manifests — as Kubernetes secrets. Changes go through helm upgrade. Paralleliq reads the release metadata to identify which values key controls the setting being changed (nodeSelector, resource limits, vLLM startup args), then generates the precise upgrade command:

helm upgrade vllm-mistral-7b ./charts/vllm \
  --reuse-values \
  --set gpu.nodeSelector.accelerator=nvidia-l4

The --reuse-values flag is non-negotiable — it scopes the change to exactly what was recommended and leaves everything else untouched.

Direct Kubernetes API

Smaller teams or clusters managed without a GitOps layer use direct kubectl or the Kubernetes API. Paralleliq's in-cluster actuator applies a targeted JSON patch to the Deployment spec. The change is immediate once approved. The audit trail lives in Paralleliq rather than in a Git commit.

---

Rollout safety is the operator's responsibility, not the tool's

Changing a Deployment in Kubernetes triggers a rolling update. A new pod starts on the target GPU tier, the serving framework loads the model into memory, the pod passes its readiness probe, and then — only then — the old pod is terminated and traffic shifts.

For stateless web services, this takes seconds. For inference workloads, the model loading window changes the calculus entirely. A 7B model at FP16 takes 15–30 seconds to load. A 70B model takes 60–120 seconds. During that window, capacity is reduced.

Two Kubernetes settings control whether this goes smoothly:

• `maxUnavailable: 0` — the old pod is never terminated until the new one is ready. This is not the Kubernetes default.
• A working readiness probe — vLLM exposes a /health endpoint. Until it returns 200, Kubernetes will not send traffic to the new pod and will not count it as available.

Without both settings, a rolling update on an inference workload can briefly serve zero replicas.

Paralleliq's role at this boundary is pre-flight checking, not pipeline ownership.

Before dispatching a change, Paralleliq checks: Is this a single-replica deployment? Is maxUnavailable set correctly? Does a PodDisruptionBudget exist? It surfaces these as warnings in the approval flow — not blockers, but signals the operator should see before authorizing the change. The CD pipeline executes the rollout. Paralleliq makes sure the operator has the full picture first.

---

The loop only closes with verification

This is the most commonly missing piece in GPU optimization workflows.

A recommendation is a hypothesis: "if you move this model to an L4, OOM kills will stop." Approving and applying the recommendation tests that hypothesis. But most tools treat the actuation event as the end of the story. The recommendation moves to "approved," the dashboard clears it, and the team moves on.

The problem: recommendations can be correct in analysis and wrong in outcome. A model moved to a larger tier might still OOM if the workload grew while the change was being rolled out. A replica scale-up might not resolve KV cache pressure if the root cause is prompt length, not pod count. Without a verification signal, you have a decision audit trail but not a feedback loop.

Post-actuation verification means watching the specific metrics that triggered the recommendation and confirming they moved in the right direction:

• OOM kills: 0 in the 24 hours following a tier upgrade
• GPU utilization: normalized from 11% to 80%+ after a tier downgrade
• KV cache utilization: below 85% following a replica scale-out

That verified signal is what turns an audit trail into a learning system. Each verified recommendation becomes a data point: this model, this cluster configuration, this traffic profile, this change — outcome confirmed.

---

What the full audit chain should look like

A complete actuation record for a GitOps environment has six entries, not two:

This chain answers every compliance question before it is asked: who decided, what changed, who reviewed the diff, when it hit the cluster, and whether it worked. It is also the chain that makes rollback unambiguous — if verification fails, you know exactly what to revert and when the pre-change state was last healthy.

---

Paralleliq is the model-aware GPU fleet optimization layer for AI infrastructure. It surfaces recommendations, routes them through your deployment integration, and verifies outcomes — without owning your cluster or replacing your control plane.

[Request a demo](https://paralleliq.ai)