A cyber range lab on the Rogue Arena canvas, with Windows machines grouped into networks.

Why We Bought a GPU for Our Cyber Range AI

November 18, 2026•9 min read

TL;DR: Tuning Oracle, the AI that builds our cyber range labs, takes thousands of test builds, and frontier-model prices were out of reach. On one rented RTX PRO 6000 we got six builds running at once for about 17 cents each. We bought one to tune all day on a model that won't refuse to build vulnerable things.

The Problem

Oracle is the AI agent inside Rogue Arena, our cyber range platform, that builds a lab from a prompt. You describe an environment and it creates the networks, adds the machines, fills Active Directory with users and groups, puts believable files on people's desktops, sets the firewall rules, and then writes curriculum for it.

It does all of that through tool calls. A tool call is the model asking our platform to do one thing, like "add a VLAN" or "create this file". A mid-sized lab takes around 200 of them.

Every change we make to a prompt or a tool description can make Oracle better at one thing and worse at another. The only way to find out is to rebuild the same test labs after each change and check the results. On October 1, one build of our main test lab cost $3.60 on Claude Sonnet.

That's fine for one build. A tuning loop wants around a hundred a day, which at that price is more than $10,000 a month. We're bootstrapped, so that was a no.

Hosting a model ourselves was the obvious thing to try. Open-weight models are ones you can download and run on your own hardware. Back in August we had watched Qwen 3.8 27B write its own C# Mythic agent in one of our red team labs, so we knew the small ones had become capable. What I didn't know was whether one could drive Oracle, or what it would cost to run. Cheaper API models were the other option, and they come back into this story near the end.

A Few Terms

If you already run local models, skip this part.

  • Token: the unit a model reads and writes, roughly four characters of text. API providers bill per token.
  • Open-weight model: a model whose weights (the billions of numbers it is made of, also called parameters) you can download and run on your own hardware, like Qwen or Gemma. "27B" means 27 billion of them.
  • Self-hosting: running the model on hardware you control instead of paying a provider for each token.
  • Inference engine: the software that loads those weights onto a GPU and answers requests. We tried three: NInfer, vLLM and SGLang.
  • Quantization: storing each weight in fewer bits (16, 8 or 4) so the model is smaller and faster, at some cost in accuracy.

The Test

We call our main test lab Northstar. The first prompt asks for a small hospital network, and a second one asks for a short curriculum to go with it:

The Northstar test lab: five networks, eight machines, 60 users, 30 files and a two-chapter curriculum.

Then automated checks look at the result. Are there exactly 60 users? Did the file server get a file-storage role? Are all 30 files in the right folders? Is every network default-deny?

A build passes only if every check passes. A lab that is 95% right still fails, which is harsh, but a student would notice the missing 5%.

Renting a Card

We didn't want to buy a GPU just to find out whether this would work, so we rented one first.

The first plan was RunPod serverless, where you upload a model and pay only while it is answering. The cards we needed (an RTX 5090 or an RTX PRO 6000) were rarely in stock for serverless, so requests sat waiting for a worker to start.

So we rented a whole RTX PRO 6000 by the hour instead: 96 GB of memory for $2.09 an hour.

The first model was Qwen3.8-27B, quantized to 4 bits, in an "abliterated" build. Abliterated means the model's refusal behaviour has been removed. A lab builder has to plant weak passwords, misconfigured services and attack paths on purpose, and I didn't want a model that pushes back on that. We need the same for RT2, our upcoming red team course, where the material is offensive by design.

Three Engines

We ran Northstar on each engine, with the same model and the same kind of card:

Minutes for one build on each engine: NInfer 27, vLLM 44 and 86, SGLang did not finish.

This was not a fair fight. NInfer was running 4-bit weights and vLLM was running 8-bit and 16-bit ones, and that alone explains a lot of the gap. One request on its own wrote about 150 tokens a second on NInfer, 35 to 39 on vLLM at 8 bits, and about 25 at 16 bits. Each engine also ran on a different rented machine (RunPod calls them pods), and pods with the same card don't all perform the same.

SGLang never finished. It holds a tool call back until the model has finished writing it, and at 16-bit speeds the one call that creates the whole Active Directory structure took longer to write than the 115 seconds of silence we allowed before dropping a connection. We now give self-hosted engines 300 seconds.

Several at Once

One build at a time isn't how the platform gets used. A class of students will all ask Oracle for something in the same few minutes, so the number I cared about was how many builds one card could run at once.

After some fixes on our side, four at once worked on NInfer. All four builds finished, in 31 minutes on average, and two passed every check.

Six at once did not go as well. Two of six passed, and the engine was producing the same five responses a minute it had managed with four, so it was full.

NInfer shares the card by taking turns between reading prompts and writing answers, and it reads one prompt at a time. Oracle's prompts were 70,000 to 85,000 tokens long, and while one was being read, every other build's writing dropped from about 120 tokens a second to about 8.

A Different Model

The card has 96 GB of memory and the 27B model needs about 20, but running three or four copies side by side wouldn't have helped. To write each token the GPU has to read the model's active weights out of memory, and how fast it can do that (memory bandwidth, about 1.6 TB a second on this card) sets the speed. Two copies of the engine would only read the same weights twice.

Speed depends on how many bytes get read per token, so I looked for a model that reads fewer.

Qwen3.8-Flash-Next is a mixture-of-experts (MoE) model. It has 125 billion parameters but uses only about 6 billion of them for any one token. That is far less to read per token than the 27B, and it scores a little better on the published coding and tool-use benchmarks.

NInfer doesn't support it. A community-maintained SGLang fork built for this card's chip does, so that's what we ran, again in a 4-bit abliterated build.

One build finished in 14.6 minutes, curriculum included. Earlier that day, on the same test, GPT-5.6 Luna had taken 16.3 and Claude Sonnet 23.5. The 27B on NInfer had taken 34.1. I didn't expect a model on a rented card to beat the APIs on time.

Then the part I cared about:

Minutes per build and model responses per minute at 1, 6 and 12 builds at once.

Six at once worked. Four of the six passed every check, counting one that our own test had wrongly failed, and the card was producing 2.5 times as many responses a minute as NInfer had with six.

Twelve at once roughly doubled the wait for about 10% more output. Engines keep a cache of prompts they have already read, so a follow-up request only costs the new part. To let more requests in we'd had to shrink that cache, and twelve 70,000-token conversations filled it, so the engine ended up re-reading whole prompts. So six at once is where we stopped for now.

Pass Rates

Four passes out of six isn't something I'd put in front of students, so we went through the failed builds one at a time. Most of the causes were on our side. Our firewall tool, for example, never said the firewall was stateful, so the model opened the reverse direction "for replies" and broke default-deny.

We fixed those over two rounds and kept the fixes general. We didn't put model names in prompts or add special cases, and we left out any fix that didn't help every model.

Six-at-once results across the fix rounds: 4, 3 and 6 of 6 passed.

After round two, all six passed. Six builds is a small sample, so I wouldn't call that a pass rate yet. We've made more fixes since, but we'd shut the rented card down before we could test them.

The Money

Here is what one Northstar build cost on each setup:

Cost per build: Claude Sonnet $3.60, GPT-5.6 Luna $0.26, GPT-6 Luna $0.20, rented GPU $0.16 to $0.17.

At six builds at a time, the rented card came to 16 or 17 cents per build attempted. That only holds if the card stays busy. Run one build at a time and it's about 50 cents.

GPT-5.6 Luna did the same build for about 26 cents and GPT-6 Luna for about 20. So on price per build, the rented GPU and the cheap API models ended up close to level, and since then we've done most of our tuning on GPT-6 Luna.

Why We Bought One

Price per build didn't justify hardware, since the cheap API models were level with the rented card. We bought a Threadripper workstation and an RTX PRO 6000 Blackwell anyway, for reasons specific to us:

  • We want tuning runs going around the clock. Six builds at a time, all day, on the cheapest API model works out to roughly $2,000 to $2,400 a month. A card you own costs the same busy or idle.
  • Renting had its own problems. Serverless stock was unreliable, and an hourly pod bills while it sits idle and often can't be restarted once you stop it.
  • We need a model that will build vulnerable things on purpose without refusing, for Oracle and for the RT2 course.

What's Next

Six builds at once is what we got working in a few days of quick tests, and I'm excited to keep going. The plan is to run the same model and engine on our own hardware, repeat the tests at six and twelve at once, add one at eight, and keep tuning the harness and prompts to see if we can reach ten or even twenty at once.

A lot of that should come from our side. After two rounds of fixes a build needed 40 to 60 model responses where our first one needed 81, and every response we cut is GPU time another build can use. Past that, it probably takes a second card.

Production requests will get the card first, and tuning runs will fill in when it's quiet. If the box is down for maintenance, we can rent a matching card for the day.

Thanks for reading. We'll have the next part of our in-platform AI saga in the next few weeks, as we keep optimizing and start testing the card on the red team tool writing in RT2.

Nick Downer

Nick Downer

Nick brings 10+ years of red team experience across DoD and commercial environments, including co-authoring RTFMv2 and building the RTFM Video Library.

Back to Blog