Run Your GPUs Like a Trading Desk: Rules Decide, Models Forecast

A trading desk hunts for mispricings but survives on its risk limits. Cutting GPU waste works the same way: rules spot the waste and check every change against hard limits, and models forecast how each change will go. Here is how the two fit together, what happens when the forecast is unsure, and what happens when traffic spikes without warning.
A trading desk exists to find mispricings: a share that costs less on one exchange than on another, a price that hasn't caught up with the news. Each one is money left on the table, and the desk is paid to pick it up.
But no desk survives on finding opportunities. It survives on risk limits: no single position bigger than a set share of the fund, no trading when the connection to the exchange is too slow to be sure an order fills, a hard stop on the day's losses. A desk with brilliant models and no limits is one bad afternoon from closing.
Running GPUs for AI has the same shape. The opportunity is waste: replicas nobody needs at this hour, a model on a bigger GPU than it uses, capacity reserved and idle. The risk is the production incident: a user-facing model that slows down or fails because someone cut too close. And the answer is the same as on the desk. Let rules spot the opportunity and check that taking it stays within limits. Let models forecast how it will go, and use that forecast to decide how it's handled.
This post walks through how that works in Paralleliq, using the desk as the guide.
The desk and the cluster
| On the trading desk | In your GPU cluster |
|---|---|
| Market data: prices, order book depth, volume | Telemetry: GPU use, queue length, KV-cache use, latency per model |
| How this stock behaved around past news | How this model's traffic peaked over the past week |
| A mispricing | Waste: idle replicas, an oversized GPU, reserved capacity nobody uses |
| A trading signal that spots it | A rule that detects the waste and names the fix |
| Pre-trade risk checks | Safety limits, checked before any change |
| A model that estimates how the trade will go | Prediction models forecasting cost, latency and throughput as ranges |
| The fund's mandate | Your policy: cost first, balanced or performance first |
| The risk officer | Your platform team |
| The trade blotter | The audit log: every change, who approved it, what happened |
| Post-trade monitoring | The rules run again after the change, to confirm it held |
How a change happens
On a desk, a model doesn't go looking for trades on its own. A signal spots the mispricing, the pre-trade checks confirm the trade is within limits, and only then does a model estimate how well that particular trade will go.
Paralleliq follows the same order:
- A rule fires on measured data. It sees evidence of waste, for example a model that has been lightly loaded all week, and it names the fix: fewer replicas, or a smaller GPU.
- The same rule checks the safety limits, on what was measured: peaks, latency, memory. If any limit fails, the change stops here.
- The models forecast that fix: what it would do to cost, latency and throughput.
- The forecast's confidence decides who acts: Paralleliq's agent, or your team.
- After the change, the rules run again on fresh data, to confirm it worked.
Steps 1, 2 and 5 run today, with every change approved by your team. Steps 3 and 4 are what we're building.
The models never invent a change. They forecast the one a rule proposed.
The limits don't negotiate
Risk limits on a desk aren't advice. If a trade would put 7% of the fund into one position and the limit is 5%, the trade doesn't happen. Nobody weighs it against how good the opportunity looks. And the check happens before the trade, using what the desk already knows: the current portfolio and the size of the order.
Paralleliq's safety limits work the same way: deterministic checks on measured facts, made before anything changes. Examples:
- Latency: the model's measured p95 latency is inside the target you set for it.
- Memory headroom: at the past week's peak, the KV cache stayed well short of full, and the GPU never swapped.
- Queueing: at peak, only a small share of requests were waiting for a slot.
The rules don't predict. They read what was measured and say yes or no. That's what makes them explainable: for any change that is proposed, you can see which conditions matched and with what values.
The forecast is a range, not a number
A good desk doesn't ask its models "will this trade make money?" It asks "how much, and how sure are you?" A model that says "between $10 and $12" is telling you something very different from one that says "somewhere between a $50 loss and a $70 gain", even if the midpoint is the same.
Paralleliq's prediction models work the same way. For the fix a rule proposed, say running a model on three replicas instead of four, they forecast cost, latency and throughput, and each forecast comes as a range: a low case, a likely case and a high case. The width of the range is the confidence. Narrow means the models have seen this kind of workload behave this way many times. Wide means they're guessing.
Your policy then sets what the forecast has to show. This is a check on the forecast, separate from the safety rules. Under cost first, the change has to save money in the likely case, and latency has to stay inside its target even in the high case: the forecast is judged on its worst plausible outcome for the thing you can't afford to lose, not on its average.
Three outcomes
Every proposed change ends in one of three places.
A limit fails: the change is dropped. On the desk: volatility jumps, and the trade would now need 7% of the fund. The pre-trade check fails, the trade doesn't happen, and nobody is paged about a trade that was never allowed. It doesn't matter how good the model thought it was: it never gets asked.
In the cluster: the model looks oversized, but at last week's peak its KV cache was already close to full. A smaller GPU would have less room, not more. Dropped: no finding is raised, so your team isn't asked to approve something that breaks a limit.
The limits pass, but the forecast is unsure: your team decides. On the desk: the order book is full of noise the models have rarely seen, and they disagree. The trade is within every limit, but nobody can say with confidence what happens next. So it goes to the risk officer's screen for a human call.
In the cluster: a newly deployed model, or a week that included a product launch, so the forecast's range is wide. The limits pass, but the change goes to your team for approval, with the forecast and the evidence attached.
The limits pass, and the forecast is confident: the agent acts. On the desk: a liquid stock reacting to an earnings report in a pattern the models have traded thousands of times. The forecast is tight, every limit is green, and the mandate allows it, so the system trades on its own at machine speed, and post-trade monitoring watches the position.
In the cluster: a steady model with a well-understood weekly pattern, a tight forecast and every limit green. This is the case where, within the policy you set, Paralleliq's agent applies the change, and the rules check it again afterwards.
| What happens | |
|---|---|
| A limit fails | Dropped: no finding is raised. The forecast is never consulted |
| Limits pass, forecast unsure | Your team decides |
| Limits pass, forecast confident | Applied within your policy, then re-checked |
The pattern is simple: rules decide whether a change can happen at all; the forecast decides how it's handled.
The flash crash: when traffic spikes without warning
On May 6, 2010, the Dow fell nearly 1,000 points in minutes and recovered most of it soon after. No model saw it coming, because nothing like it was in the data. What protected markets afterwards wasn't better forecasting. It was circuit breakers: rules that act on what the price is doing right now, without asking any model what it thinks.
GPU clusters have their own flash crashes. A launch goes viral, a batch job starts sending traffic to a serving endpoint, a client bug retries every failed request at once. Traffic multiplies faster than any forecast can follow.
Two things protect you.
Don't trade close to the limit. Every sizing decision in Paralleliq is judged against a week of peaks, not averages. A model that averages 30% of its capacity but peaks at 85% every weekday at 10am is sized for the peak. Removing waste never means leaving a service with only enough capacity for an average day.
A reflex for the spike nobody predicted. For a spike that's bigger than anything in the history, the answer is the circuit breaker, not the forecast. This is where Paralleliq is going: a reflex that watches raw signals, such as queue length and how fast traffic is rising, and protects the service first. It holds back excess load at the front door, and it adds capacity when traffic is rising fast, not after the GPUs are already saturated. Afterwards, what happened becomes data for the next forecast, so a spike that was unprecedented once is a known pattern the next time.
Earning autonomy
A new trader starts with small limits. As their record builds, the limits grow. Nobody hands a first-week hire the whole book.
Paralleliq works the same way. Today a person approves every change Paralleliq proposes, for two reasons: safety, and so Paralleliq learns your workloads before it acts on its own. Each approved change and its measured outcome sharpen the next forecast. As forecasts get tighter for a workload, more of its changes land in the "limits pass, forecast confident" case, and your policy decides how many of those may run on their own. The rules never relax along the way. Only the human checkpoint moves.
Where the analogy stops
Markets are adversarial: someone on the other side of the trade is trying to beat you. Your traffic isn't. That makes the cluster's patterns more stable than a market's, which is why forecasting them works.
And the cost of a mistake is different. A bad trade loses money. A bad GPU change makes real users wait. That's why the defaults lean further towards safety than most desks would: when in doubt, ask a person; when a limit fails, don't act.
What runs today
Today Paralleliq collects telemetry from your cluster, runs its rules against a week of peaks, prices each finding in dollars per month, routes it to your team for approval, and records every step. The prediction models, the policies, the agent's automatic path and the spike reflex are the direction we're building in, in that spirit: rules decide, models forecast, and autonomy is earned.
If you'd like to see what your own cluster's waste looks like through these rules, start with a Lightweight Scan.