A safety guardrail for jailbreaks, prompt injections, and content moderation that runs in single-digit milliseconds on commodity CPUs — powered by Resource Aware Attention, a new attention mechanism designed for the deployment envelope.
Every production LLM call now passes through one or more safety classifiers — jailbreak detectors, prompt-injection detectors, content moderators. They run on every request, on every input and again on every output, almost always on CPU fleets.
Today the entire category sits in the hundreds-of-milliseconds-per-classification regime, with input lengths capped at 512 tokens, and forces operators to choose between guardrails that miss most attacks and guardrails that block most benign traffic. Compression has not closed the gap.
We introduce Bud Sentinel, a CPU-native safety guardrail for large language models. It covers jailbreak detection, prompt-injection detection, and content moderation across toxicity, hate, harassment, self-harm, violence, illegal, and regulated-advice categories — and is designed to run on the same fleet the application already runs on, at the latency budget the application already has.
Bud Sentinel is the only model in our evaluation that lands both Attack Success Rate and False Refusal Rate under 20% — the only evaluated model that sits on a deployable operating point. It delivers single-digit-millisecond classification on commodity CPUs, faster than every transformer baseline we evaluated, including those same baselines on an A100 GPU.
These numbers are not the product of incremental compression. Bud Sentinel is built on Resource Aware Attention (RAA) — a new attention mechanism designed from the ground up against the resource envelope of the deployment target (CPU cache hierarchy, precision tier, latency SLO), with the envelope as a first-class input to the mechanism rather than a downstream concern. Bud Sentinel is the first product in a broader family of RAA models described in this paper.
The small-model tax on the LLM stack, the deployment envelope that current guardrails ignore, and why another round of compression will not close the gap.
A jailbreak detector decides whether the input is admissible. A prompt-injection detector decides whether retrieved context is trying to take over the model. A content moderator decides whether the generation is safe to ship. These classifiers run on every request — almost always on the same CPU fleets the application already runs on, often on shared tenants, often with an L3 cache already under pressure from co-resident workloads.
This deployment envelope is not what current safety guardrails were designed for. Existing transformer-based guard models are built from the same architectural template as the generative model the operator is trying to protect: a transformer trained on GPU and deployed by trying to compress it until it fits on a CPU. The template works — but it also leaves the entire safety-classifier category in the hundreds-of-milliseconds-per-classification regime on CPU, capped at 512 tokens, forcing a choice between guardrails that miss most attacks and guardrails that block most benign traffic.
These numbers are not what another round of compression on the existing template would produce. They are the result of designing an attention mechanism from the deployment envelope inward — with the cache hierarchy, the precision tier, and the latency SLO treated as first-class inputs to the mechanism itself. We call it Resource Aware Attention (RAA), and Bud Sentinel is the first product built on it.
Across the public benchmarks we evaluate against, Bud Sentinel delivers single-digit-millisecond classification on commodity CPUs — faster than every transformer baseline we measured, including the same baselines running on an A100 GPU — while sustaining over 4,400 req/s on a server-class Xeon node at 512 tokens and handling inputs as large as 65,536 tokens natively.
The cost of small models on the LLM stack has three compounding properties that make it urgent to solve at the mechanism level rather than the compression level.
A generative call amortizes its cost across a long output. A safety classifier pays its cost once per input and again per output; a retrieval scorer pays once per candidate. These workloads have no natural batching geometry and cannot hide latency behind token streaming.
Operators do not put a dedicated GPU in front of every safety classifier — they cannot afford to, and the call graph would not let them saturate it. These classifiers run on CPU fleets, on shared tenants, with the L3 already under pressure. GPU attention's cost profile is simply the wrong profile.
Shrinking, quantizing to int8, exporting to faster runtimes — these wins are real but have not moved the category out of the hundreds-of-milliseconds regime. Current small transformer guards land between 334 ms and 3,854 ms per classification on the CPUs operators actually run on.
Transformer guards almost universally cap input at 512 tokens — a direct consequence of the quadratic cost of standard self-attention. Long transcripts, multi-turn history, retrieved RAG context, uploaded documents, and code files each become a separate engineering problem (chunking, voting, re-aggregation) that degrades both accuracy and latency further.
The right move is not to compress the existing mechanism further. It is to design attention differently — with CPU inference as a first-class constraint of the mechanism itself, and long inputs as a native capability rather than a retrofit.
The mechanism behind Bud Sentinel — designed against the resource envelope from the start — and the family of models built on it.
RAA is the attention mechanism behind Bud Sentinel. It is not an attempt to match a frontier transformer on every benchmark — it is a deliberately different point on the accuracy-versus-resource curve, and the curve looks fundamentally different once the resource envelope is built into the mechanism rather than bolted on afterward.
The target cache hierarchy, the precision tier the runtime executes in, and the latency SLO the application commits to are all declared before training. A model trained for one envelope is not retargeted through post-hoc compression; a new envelope is a new training run. The resources are not something the mechanism tolerates — they are something it is shaped by.
RAA's attention does not suffer the quadratic growth of standard self-attention. Efficient-attention work — Linformer, Performer, Longformer — has shown subquadratic attention is achievable, but those mechanisms were designed first and resourced later. RAA inverts that order: long inputs cause the mechanism to do more of the same cheap work; they do not cause a phase change in cost.
A single RAA pass supports many downstream task heads in parallel — a safety decision, a PII span extraction, an intent classification, a routing decision. The marginal cost of another head is the head's own work, not another attention pass. For systems already chaining five or six small classifiers per request, RAA turns that chain into one shared attention cost with many cheap decisions on top.
RAA is a mechanism; the RAA family is the first line of models built on it. We organize the family by the task class and resource envelope each variant targets, rather than by parameter count.
Targets a standard server-CPU envelope; supports classification, scoring, span extraction, and retrieval-style tasks with equal first-class standing. The foundation most task-specific variants are built from.
Trained against the jailbreak, prompt-injection, and content-moderation surfaces that sit on the input and output edges of an LLM. Powers the Bud Sentinel product line.
Tuned for PII extraction, compliance tagging, code detection, and secret scanning — workloads that need attribution, not just a score.
A retrieval-oriented variant designed for the resource envelope of a retrieval pipeline rather than a classifier.
Targets a tighter envelope than RAA-Base and a sub-millisecond SLO.
Because every variant shares the same mechanism, multiple variants can be co-hosted in a single process and share one attention pass per request. A gateway running separate safety, router, and PII transformers — each at 100–900 ms on CPU — can in principle replace all three with a single-process RAA deployment whose per-request cost is dominated by one attention pass, regardless of how many heads it carries.
The guardrail product line built on RAA-Safety, and the accuracy, latency, and throughput numbers measured against public benchmarks and public competitor models.
Bud Sentinel is the safety-guardrail product line built on RAA-Safety — jailbreak detection, prompt-injection detection, and content moderation. It is designed to deploy on the same fleet the application already runs on, at the latency budget it already has, not in a separate GPU service with its own scaling story.
A safety classifier that adds 300 ms to every turn changes the shape of an application; one that adds 8 ms does not. That difference is the whole case.
We evaluate on an aggregated benchmark drawn from four widely used suites — JailBreakBench, PIGuard, WildJailbreak, and the Qualifire Prompt Injection Benchmark — reporting Attack Success Rate (ASR, the probability a harmful prompt gets through) and False Refusal Rate (FRR, the probability a benign prompt is blocked).
| Model | ASR | FRR |
|---|---|---|
| Bud Sentinel-Jailbreak | 15.97% | 14.92% |
| Prompt-Guard-2-86M | 34.68% | 15.30% |
| Prompt-Guard-86M | 5.83% | 89.35% |
| ArchGuard | 5.40% | 81.65% |
| PIGuard | 25.01% | 25.86% |
| ProtectAI PI V2 | 36.35% | 24.39% |
Bud Sentinel-Jailbreak is the only model with both ASR and FRR below 20%. The two models that show lower ASR (Prompt-Guard-86M, ArchGuard) achieve it by refusing almost everything — 89% and 82% false-refusal rates. No product team can ship those operating points. On the deployable frontier, Bud Sentinel is not merely the leader; in our evaluation it is the only occupant.
On the harder individual slices the pattern holds, with the strongest single result on WildJailbreak: ASR 6.5%, FRR 11.42% — simultaneously lower false-negative and false-positive rates than every other evaluated model on that benchmark.
All models evaluated at 512-token sequence length — the maximum most transformer baselines support. Competitors measured via their published runtime pipelines on the bare machine; Bud Sentinel measured end-to-end over its serving interface, so its numbers include the network and RPC path a real client request would traverse.
| Model | i7-11370H | A100-80GB | EPYC 7V13 | Xeon 8272CL |
|---|---|---|---|---|
| Bud Sentinel-Jailbreak | 8.39 ms | n/a | 5.67 ms | 5.99 ms |
| Prompt-Guard-2-86M | 803.85 ms | 18.52 ms | 3854.91 ms | 334.15 ms |
| Prompt-Guard-86M | 883.99 ms | 18.92 ms | 3871.68 ms | 401.97 ms |
| ArchGuard | 850.43 ms | 19.07 ms | 3851.31 ms | 379.89 ms |
| PIGuard | 844.18 ms | 19.00 ms | 3777.85 ms | 349.33 ms |
| ProtectAI PI V2 | 841.37 ms | 19.13 ms | 3842.30 ms | 345.19 ms |
The deployment question operators ask is what a single node sustains under load. We report three envelopes: a modern server-class Xeon, a commodity Xeon for long-sequence workloads, and a fanless laptop-class CPU.
| Input | Conc. | p50 | p95 | p99 | req/s |
|---|---|---|---|---|---|
| 128 tok | 1 | 2.57 | 3.04 | 3.11 | 341 |
| 128 tok | 10 | 3.99 | 5.03 | 5.25 | 1,937 |
| 128 tok | 50 | 7.00 | 10.45 | 11.35 | 4,690 |
| 512 tok | 1 | 3.05 | 3.25 | 3.28 | 330 |
| 512 tok | 10 | 4.57 | 5.30 | 5.59 | 1,809 |
| 512 tok | 50 | 7.96 | 10.78 | 11.16 | 4,445 |
| At 50-way concurrency a single Xeon 6972P sustains 4,400+ req/s at 512 tokens with a p99 below 12 ms — a safety classifier on every request that will never be the bottleneck. | |||||
| Seq len | p10 | p50 | p90 | p99 | req/s |
|---|---|---|---|---|---|
| 512 | 9.29 | 25.17 | 34.93 | 44.16 | 2,704 |
| 8,192 | 67.05 | 151.07 | 214.06 | 262.75 | 437 |
| 16,384 | 97.97 | 254.73 | 404.86 | 571.22 | 235 |
| 65,536 | 197.00 | 559.98 | 933.79 | 1214.18 | 100 |
| The same model classifies a 65,536-token input at 560 ms p50 — an input length the transformer baselines (capped at 512 tokens) do not operate at all. | |||||
| Input | Concurrency | p50 | req/s |
|---|---|---|---|
| 128 tok | 1 (low load) | 1.18 | 746 |
| 128 tok | 50 (high load) | 22.99 | 1,493 |
| 512 tok | 1 (low load) | 1.80 | 498 |
| 512 tok | 50 (high load) | 26.32 | 881 |
| On a fanless ultraportable, Bud Sentinel sustains nearly 1,500 req/s at 128 tokens — guardrails that run inside a desktop app, a browser extension, an IDE, or an embedded agent, with no server call at all. | |||
What RAA is not, the layered product Bud Sentinel ships into, where the mechanism goes beyond safety, and what we plan to publish next.
Bud Sentinel ships as a layered product. The RAA-powered fast layer handles the firehose at single-digit-millisecond latency; behind it sit deeper scans — higher-capacity review for tail risk, specialized detectors for adversarial patterns, and escalation paths for ambiguous cases. The speed of the fast layer is what makes the deeper layers affordable to run at production volume.
RAA is an attention mechanism, not a guardrail trick. Any CPU-deployed small-model workload in the modern LLM stack is a candidate:
RAA variants targeted at progressively tighter envelopes (single-core, mobile, embedded) and richer tasks (retrieval, multi-hop classification, span linking) — so "what envelope are you deploying into?" is answered by picking a family member, not running a compression project.
Running several variants together in one process over a shared attention pass: a single-process deployment carrying a full guardrail, routing, and span-extraction stack for the per-request cost of any one of them today.
Harder, less-gameable jailbreak and prompt-injection benchmarks built around the deployable-operating-point framing — updated as the field moves, with direct reporting of regressions.
The resource envelope has been an afterthought in attention design for too long. Architectures are chosen on GPUs and taped together with compression tricks on their way to CPU. In most of the modern LLM stack — every small model on every request — that is the wrong order of operations.
Bud Sentinel is the first concrete demonstration that flipping that order pays off. A guardrail that classifies in single-digit milliseconds on commodity CPUs, sustains thousands of requests per second on a single node, runs on a fanless laptop, and natively handles 65,536-token inputs is not a compression result — it is a different curve. Resource Aware Attention is a bet that this curve generalizes well beyond safety, to every CPU-deployed small-model workload in the stack.
We would like to talk to teams operating at the edges where this matters — safety pipelines, retrieval stacks, gateways, on-device deployments. If that is you, please reach out: {adarsh.ms, dittops, jithinvg}@bud.studio
All competitor models evaluated through their published inference pipelines locally; Bud Sentinel evaluated end-to-end over its serving interface. All at 512-token sequence length unless noted. ASR is the false-negative rate on the harmful class; FRR the false-positive rate on the benign class.
| Model | ASR | FRR |
|---|---|---|
| Bud Sentinel-Jailbreak | 16.20% | 47.00% |
| Prompt-Guard-2-86M | 27.94% | 17.00% |
| Prompt-Guard-86M | 2.63% | 98.00% |
| ArchGuard | 1.00% | 100.00% |
| PIGuard | 18.36% | 26.00% |
| ProtectAI PI V2 | 34.10% | 1.00% |
| Model | ASR | FRR |
|---|---|---|
| Bud Sentinel-Jailbreak | 50.40% | 14.96% |
| Prompt-Guard-2-86M | 99.20% | 12.29% |
| Prompt-Guard-86M | 36.00% | 73.20% |
| ArchGuard | 13.60% | 87.40% |
| PIGuard | 37.60% | 20.68% |
| ProtectAI PI V2 | 89.60% | 29.46% |
| Model | ASR | FRR |
|---|---|---|
| Bud Sentinel-Jailbreak | 6.50% | 11.42% |
| Prompt-Guard-2-86M | 56.30% | 28.50% |
| Prompt-Guard-86M | 8.50% | 91.40% |
| ArchGuard | 14.20% | 72.38% |
| PIGuard | 39.25% | 36.19% |
| ProtectAI PI V2 | 40.20% | 32.85% |
| Model | ASR | FRR |
|---|---|---|
| Bud Sentinel-Jailbreak | 18.20% | 5.56% |
| Prompt-Guard-2-86M | 36.90% | 14.36% |
| Prompt-Guard-86M | 7.90% | 93.90% |
| ArchGuard | 9.65% | 75.14% |
| PIGuard | 32.60% | 22.89% |
| ProtectAI PI V2 | 34.40% | 23.40% |
| Seq len | Conc. | p50 | p99 | req/s |
|---|---|---|---|---|
| 512 | 100 | 25.17 | 44.16 | 2,704 |
| 512 | 5,000 | 1,027.03 | 1,867.63 | 2,674 |
| 512 | 10,000 | 1,910.90 | 3,664.11 | 2,749 |
| 8,192 | 100 | 151.07 | 262.75 | 437 |
| 8,192 | 10,000 | 9,935.21 | 19,434.16 | 508 |
| 16,384 | 100 | 254.73 | 571.22 | 235 |
| 16,384 | 10,000 | 17,989.47 | 34,860.60 | 283 |
| 65,536 | 100 | 559.98 | 1,214.18 | 100 |
| 65,536 | 10,000 | 50,874.37 | 97,428.30 | 101 |