Bud Technical Whitepaper
Bud Sentinel · Resource Aware Attention

Bud Sentinel: a CPU-native safety guardrail for large language models

A safety guardrail for jailbreaks, prompt injections, and content moderation that runs in single-digit milliseconds on commodity CPUs — powered by Resource Aware Attention, a new attention mechanism designed for the deployment envelope.

5.67ms
Per-request latency on server CPU
4,400+
Requests/sec, single Xeon node
65,536
Tokens classified natively
<20%
Only model under both ASR & FRR
AuthorsAdarsh M.S. · Ditto P.S. · Jithin V.G.
DateApril 2026
Bud Sentinel · CPU-Native Safety Guardrail Bud
Abstract

A guardrail built for the fleet it runs on.

Every production LLM call now passes through one or more safety classifiers — jailbreak detectors, prompt-injection detectors, content moderators. They run on every request, on every input and again on every output, almost always on CPU fleets.

Today the entire category sits in the hundreds-of-milliseconds-per-classification regime, with input lengths capped at 512 tokens, and forces operators to choose between guardrails that miss most attacks and guardrails that block most benign traffic. Compression has not closed the gap.

We introduce Bud Sentinel, a CPU-native safety guardrail for large language models. It covers jailbreak detection, prompt-injection detection, and content moderation across toxicity, hate, harassment, self-harm, violence, illegal, and regulated-advice categories — and is designed to run on the same fleet the application already runs on, at the latency budget the application already has.

The headline result

Bud Sentinel is the only model in our evaluation that lands both Attack Success Rate and False Refusal Rate under 20% — the only evaluated model that sits on a deployable operating point. It delivers single-digit-millisecond classification on commodity CPUs, faster than every transformer baseline we evaluated, including those same baselines on an A100 GPU.

single-digit ms
Classification latency on commodity CPU
4,400+
req/s, server-class Xeon @ 512 tok
~1,500
req/s on a fanless laptop CPU
65,536
token inputs, native, on a 16-vCPU box

These numbers are not the product of incremental compression. Bud Sentinel is built on Resource Aware Attention (RAA) — a new attention mechanism designed from the ground up against the resource envelope of the deployment target (CPU cache hierarchy, precision tier, latency SLO), with the envelope as a first-class input to the mechanism rather than a downstream concern. Bud Sentinel is the first product in a broader family of RAA models described in this paper.

Bud Ecosystem · Simplifying Intelligence01
Bud Sentinel · CPU-Native Safety Guardrail Bud
Chapter 01

Why this is the right problem now

The small-model tax on the LLM stack, the deployment envelope that current guardrails ignore, and why another round of compression will not close the gap.

Chapter 01 · Why this is the right problem now Bud
Introduction

Safety classifiers run on CPU, on every call.

A jailbreak detector decides whether the input is admissible. A prompt-injection detector decides whether retrieved context is trying to take over the model. A content moderator decides whether the generation is safe to ship. These classifiers run on every request — almost always on the same CPU fleets the application already runs on, often on shared tenants, often with an L3 cache already under pressure from co-resident workloads.

This deployment envelope is not what current safety guardrails were designed for. Existing transformer-based guard models are built from the same architectural template as the generative model the operator is trying to protect: a transformer trained on GPU and deployed by trying to compress it until it fits on a CPU. The template works — but it also leaves the entire safety-classifier category in the hundreds-of-milliseconds-per-classification regime on CPU, capped at 512 tokens, forcing a choice between guardrails that miss most attacks and guardrails that block most benign traffic.

§1A different starting point

These numbers are not what another round of compression on the existing template would produce. They are the result of designing an attention mechanism from the deployment envelope inward — with the cache hierarchy, the precision tier, and the latency SLO treated as first-class inputs to the mechanism itself. We call it Resource Aware Attention (RAA), and Bud Sentinel is the first product built on it.

The shape of the result

Across the public benchmarks we evaluate against, Bud Sentinel delivers single-digit-millisecond classification on commodity CPUs — faster than every transformer baseline we measured, including the same baselines running on an A100 GPU — while sustaining over 4,400 req/s on a server-class Xeon node at 512 tokens and handling inputs as large as 65,536 tokens natively.

Bud Ecosystem · Simplifying Intelligence02
Chapter 01 · Why this is the right problem now Bud

§2The small-model tax

The cost of small models on the LLM stack has three compounding properties that make it urgent to solve at the mechanism level rather than the compression level.

1

The tax is per-request, not per-session

A generative call amortizes its cost across a long output. A safety classifier pays its cost once per input and again per output; a retrieval scorer pays once per candidate. These workloads have no natural batching geometry and cannot hide latency behind token streaming.

2

The deployment target is not a GPU

Operators do not put a dedicated GPU in front of every safety classifier — they cannot afford to, and the call graph would not let them saturate it. These classifiers run on CPU fleets, on shared tenants, with the L3 already under pressure. GPU attention's cost profile is simply the wrong profile.

3

Compression has plateaued

Shrinking, quantizing to int8, exporting to faster runtimes — these wins are real but have not moved the category out of the hundreds-of-milliseconds regime. Current small transformer guards land between 334 ms and 3,854 ms per classification on the CPUs operators actually run on.

The input-length ceiling

Transformer guards almost universally cap input at 512 tokens — a direct consequence of the quadratic cost of standard self-attention. Long transcripts, multi-turn history, retrieved RAG context, uploaded documents, and code files each become a separate engineering problem (chunking, voting, re-aggregation) that degrades both accuracy and latency further.

The right move is not to compress the existing mechanism further. It is to design attention differently — with CPU inference as a first-class constraint of the mechanism itself, and long inputs as a native capability rather than a retrofit.

Bud Ecosystem · Simplifying Intelligence03
Bud Sentinel · CPU-Native Safety Guardrail Bud
Chapter 02

Resource Aware Attention

The mechanism behind Bud Sentinel — designed against the resource envelope from the start — and the family of models built on it.

Chapter 02 · Resource Aware Attention Bud
The mechanism

Three principles govern RAA.

RAA is the attention mechanism behind Bud Sentinel. It is not an attempt to match a frontier transformer on every benchmark — it is a deliberately different point on the accuracy-versus-resource curve, and the curve looks fundamentally different once the resource envelope is built into the mechanism rather than bolted on afterward.

1 · The resource envelope is an input, not an output

The target cache hierarchy, the precision tier the runtime executes in, and the latency SLO the application commits to are all declared before training. A model trained for one envelope is not retargeted through post-hoc compression; a new envelope is a new training run. The resources are not something the mechanism tolerates — they are something it is shaped by.

2 · The attention step is bounded by the envelope, not by sequence length

RAA's attention does not suffer the quadratic growth of standard self-attention. Efficient-attention work — Linformer, Performer, Longformer — has shown subquadratic attention is achievable, but those mechanisms were designed first and resourced later. RAA inverts that order: long inputs cause the mechanism to do more of the same cheap work; they do not cause a phase change in cost.

3 · Heads compose freely over a single attention pass

A single RAA pass supports many downstream task heads in parallel — a safety decision, a PII span extraction, an intent classification, a routing decision. The marginal cost of another head is the head's own work, not another attention pass. For systems already chaining five or six small classifiers per request, RAA turns that chain into one shared attention cost with many cheap decisions on top.

Bud Ecosystem · Simplifying Intelligence04
Chapter 02 · Resource Aware Attention Bud

§4The RAA family of models

RAA is a mechanism; the RAA family is the first line of models built on it. We organize the family by the task class and resource envelope each variant targets, rather than by parameter count.

RAA-BaseAvailable

The general-purpose backbone

Targets a standard server-CPU envelope; supports classification, scoring, span extraction, and retrieval-style tasks with equal first-class standing. The foundation most task-specific variants are built from.

RAA-SafetyAvailable

The safety-specialized variant

Trained against the jailbreak, prompt-injection, and content-moderation surfaces that sit on the input and output edges of an LLM. Powers the Bud Sentinel product line.

RAA-SpanRoadmap

Token- and span-level extraction

Tuned for PII extraction, compliance tagging, code detection, and secret scanning — workloads that need attribution, not just a score.

RAA-RetrieveRoadmap

Re-ranking at candidate-set scale

A retrieval-oriented variant designed for the resource envelope of a retrieval pipeline rather than a classifier.

RAA-RouteRoadmap

Request classification at the gateway edge

Targets a tighter envelope than RAA-Base and a sub-millisecond SLO.

Designed to run together

Because every variant shares the same mechanism, multiple variants can be co-hosted in a single process and share one attention pass per request. A gateway running separate safety, router, and PII transformers — each at 100–900 ms on CPU — can in principle replace all three with a single-process RAA deployment whose per-request cost is dominated by one attention pass, regardless of how many heads it carries.

Bud Ecosystem · Simplifying Intelligence05
Bud Sentinel · CPU-Native Safety Guardrail Bud
Chapter 03

Bud Sentinel: design & results

The guardrail product line built on RAA-Safety, and the accuracy, latency, and throughput numbers measured against public benchmarks and public competitor models.

Chapter 03 · Bud Sentinel: design & results Bud
Design & deployment

Between a user and a model.

Bud Sentinel is the safety-guardrail product line built on RAA-Safety — jailbreak detection, prompt-injection detection, and content moderation. It is designed to deploy on the same fleet the application already runs on, at the latency budget it already has, not in a separate GPU service with its own scaling story.

A safety classifier that adds 300 ms to every turn changes the shape of an application; one that adds 8 ms does not. That difference is the whole case.

§6.1Accuracy on jailbreak & prompt-injection

We evaluate on an aggregated benchmark drawn from four widely used suites — JailBreakBench, PIGuard, WildJailbreak, and the Qualifire Prompt Injection Benchmark — reporting Attack Success Rate (ASR, the probability a harmful prompt gets through) and False Refusal Rate (FRR, the probability a benign prompt is blocked).

Table 1 — Overall aggregate accuracy across all four benchmarks
ModelASRFRR
Bud Sentinel-Jailbreak15.97%14.92%
Prompt-Guard-2-86M34.68%15.30%
Prompt-Guard-86M5.83%89.35%
ArchGuard5.40%81.65%
PIGuard25.01%25.86%
ProtectAI PI V236.35%24.39%
The deployable operating point

Bud Sentinel-Jailbreak is the only model with both ASR and FRR below 20%. The two models that show lower ASR (Prompt-Guard-86M, ArchGuard) achieve it by refusing almost everything — 89% and 82% false-refusal rates. No product team can ship those operating points. On the deployable frontier, Bud Sentinel is not merely the leader; in our evaluation it is the only occupant.

On the harder individual slices the pattern holds, with the strongest single result on WildJailbreak: ASR 6.5%, FRR 11.42% — simultaneously lower false-negative and false-positive rates than every other evaluated model on that benchmark.

Bud Ecosystem · Simplifying Intelligence06
Chapter 03 · Bud Sentinel: design & results Bud

§6.2Per-request latency on CPU

All models evaluated at 512-token sequence length — the maximum most transformer baselines support. Competitors measured via their published runtime pipelines on the bare machine; Bud Sentinel measured end-to-end over its serving interface, so its numbers include the network and RPC path a real client request would traverse.

Table 2 — Per-request latency at 512 tokens across CPU and GPU targets
Modeli7-11370HA100-80GBEPYC 7V13Xeon 8272CL
Bud Sentinel-Jailbreak8.39 msn/a5.67 ms5.99 ms
Prompt-Guard-2-86M803.85 ms18.52 ms3854.91 ms334.15 ms
Prompt-Guard-86M883.99 ms18.92 ms3871.68 ms401.97 ms
ArchGuard850.43 ms19.07 ms3851.31 ms379.89 ms
PIGuard844.18 ms19.00 ms3777.85 ms349.33 ms
ProtectAI PI V2841.37 ms19.13 ms3842.30 ms345.19 ms
Bud Sentinel
5.67 ms
PIGuard
3,777 ms
ProtectAI PI V2
3,842 ms
ArchGuard
3,851 ms
Prompt-Guard-2-86M
3,855 ms
Figure 1. Per-request latency on the EPYC 7V13 server CPU operators actually run on. Bud Sentinel's bar is a 0.15% sliver of the transformer baselines — a gap of two-to-three orders of magnitude on the same silicon.
8.39ms
Bud Sentinel on a laptop CPU, end-to-end — faster than every transformer peer running on an A100 (18–19 ms).
678×
Faster than the baselines on the same EPYC 7V13 server CPU (5.67 ms vs ~3,840 ms).
Bud Ecosystem · Simplifying Intelligence07
Chapter 03 · Bud Sentinel: design & results Bud

§6.3Production throughput on CPU

The deployment question operators ask is what a single node sustains under load. We report three envelopes: a modern server-class Xeon, a commodity Xeon for long-sequence workloads, and a fanless laptop-class CPU.

Server class — Intel Xeon 6972P
InputConc.p50p95p99req/s
128 tok12.573.043.11341
128 tok103.995.035.251,937
128 tok507.0010.4511.354,690
512 tok13.053.253.28330
512 tok104.575.305.591,809
512 tok507.9610.7811.164,445
At 50-way concurrency a single Xeon 6972P sustains 4,400+ req/s at 512 tokens with a p99 below 12 ms — a safety classifier on every request that will never be the bottleneck.
Long-sequence class — Intel Xeon Platinum 8272CL (concurrency 100)
Seq lenp10p50p90p99req/s
5129.2925.1734.9344.162,704
8,19267.05151.07214.06262.75437
16,38497.97254.73404.86571.22235
65,536197.00559.98933.791214.18100
The same model classifies a 65,536-token input at 560 ms p50 — an input length the transformer baselines (capped at 512 tokens) do not operate at all.
Laptop / on-device — Intel Core Ultra 7 268V (Lunar Lake)
InputConcurrencyp50req/s
128 tok1 (low load)1.18746
128 tok50 (high load)22.991,493
512 tok1 (low load)1.80498
512 tok50 (high load)26.32881
On a fanless ultraportable, Bud Sentinel sustains nearly 1,500 req/s at 128 tokens — guardrails that run inside a desktop app, a browser extension, an IDE, or an embedded agent, with no server call at all.
Bud Ecosystem · Simplifying Intelligence08
Bud Sentinel · CPU-Native Safety Guardrail Bud
Chapter 04

Trade-offs, reach & roadmap

What RAA is not, the layered product Bud Sentinel ships into, where the mechanism goes beyond safety, and what we plan to publish next.

Chapter 04 · Trade-offs, reach & roadmap Bud
Honesty about the curve

What RAA is, and is not.

What RAA is not

Not a generative model

  • A mechanism for classification, scoring, and span extraction
  • Not a replacement for a frontier LLM on open-ended reasoning
  • Will not match a top transformer's absolute-accuracy ceiling given unbounded compute
What RAA is

The most accurate deployable model at the envelope

  • The models that beat its raw ASR do so only at unshippable false-refusal rates
  • Harder slices (PIGuard) are harder for everyone, ours included — we say so directly
  • A conscious trade of expressive ceiling for resource discipline
The RAA layer is one of several

Bud Sentinel ships as a layered product. The RAA-powered fast layer handles the firehose at single-digit-millisecond latency; behind it sit deeper scans — higher-capacity review for tail risk, specialized detectors for adversarial patterns, and escalation paths for ambiguous cases. The speed of the fast layer is what makes the deeper layers affordable to run at production volume.

§8Applications beyond guardrails

RAA is an attention mechanism, not a guardrail trick. Any CPU-deployed small-model workload in the modern LLM stack is a candidate:

  • Retrieval & re-ranking — score thousands of candidate passages per query on CPU within a gateway's latency budget.
  • Routing & model selection — a millisecond-class router strictly dominates a hundred-millisecond transformer router most gateways cannot afford.
  • Entity, PII & compliance extraction — span-level tagging on the same node as the application, alongside the safety pass already running.
  • Long-context classification — 65,536-token inputs as a single call instead of a chunking-and-voting pipeline.
  • Edge & on-device — guardrails inside a desktop app, browser extension, IDE, offline assistant, or embedded gateway.
Bud Ecosystem · Simplifying Intelligence09
Chapter 04 · Trade-offs, reach & roadmap Bud

§9Research directions

1

More variants, more envelopes

RAA variants targeted at progressively tighter envelopes (single-core, mobile, embedded) and richer tasks (retrieval, multi-hop classification, span linking) — so "what envelope are you deploying into?" is answered by picking a family member, not running a compression project.

2

Multi-variant co-hosting

Running several variants together in one process over a shared attention pass: a single-process deployment carrying a full guardrail, routing, and span-extraction stack for the per-request cost of any one of them today.

3

Evaluation infrastructure

Harder, less-gameable jailbreak and prompt-injection benchmarks built around the deployable-operating-point framing — updated as the field moves, with direct reporting of regressions.

§10Closing

The resource envelope has been an afterthought in attention design for too long. Architectures are chosen on GPUs and taped together with compression tricks on their way to CPU. In most of the modern LLM stack — every small model on every request — that is the wrong order of operations.

Bud Sentinel is the first concrete demonstration that flipping that order pays off. A guardrail that classifies in single-digit milliseconds on commodity CPUs, sustains thousands of requests per second on a single node, runs on a fanless laptop, and natively handles 65,536-token inputs is not a compression result — it is a different curve. Resource Aware Attention is a bet that this curve generalizes well beyond safety, to every CPU-deployed small-model workload in the stack.

Get in touch

We would like to talk to teams operating at the edges where this matters — safety pipelines, retrieval stacks, gateways, on-device deployments. If that is you, please reach out: {adarsh.ms, dittops, jithinvg}@bud.studio

Bud Ecosystem · Simplifying Intelligence10
Appendix · Full benchmark tables Bud
Appendix A.1

Accuracy — full per-benchmark tables.

All competitor models evaluated through their published inference pipelines locally; Bud Sentinel evaluated end-to-end over its serving interface. All at 512-token sequence length unless noted. ASR is the false-negative rate on the harmful class; FRR the false-positive rate on the benign class.

JailBreakBench — Benign 100, Non-Benign 1,900
ModelASRFRR
Bud Sentinel-Jailbreak16.20%47.00%
Prompt-Guard-2-86M27.94%17.00%
Prompt-Guard-86M2.63%98.00%
ArchGuard1.00%100.00%
PIGuard18.36%26.00%
ProtectAI PI V234.10%1.00%
PIGuard — Benign 1,310, Non-Benign 125
ModelASRFRR
Bud Sentinel-Jailbreak50.40%14.96%
Prompt-Guard-2-86M99.20%12.29%
Prompt-Guard-86M36.00%73.20%
ArchGuard13.60%87.40%
PIGuard37.60%20.68%
ProtectAI PI V289.60%29.46%
WildJailbreak — Benign 210, Non-Benign 2,000
ModelASRFRR
Bud Sentinel-Jailbreak6.50%11.42%
Prompt-Guard-2-86M56.30%28.50%
Prompt-Guard-86M8.50%91.40%
ArchGuard14.20%72.38%
PIGuard39.25%36.19%
ProtectAI PI V240.20%32.85%
Qualifire Prompt Injection — Benign 3,001, Non-Benign 1,999
ModelASRFRR
Bud Sentinel-Jailbreak18.20%5.56%
Prompt-Guard-2-86M36.90%14.36%
Prompt-Guard-86M7.90%93.90%
ArchGuard9.65%75.14%
PIGuard32.60%22.89%
ProtectAI PI V234.40%23.40%
Bud Ecosystem · Simplifying Intelligence11
Appendix · Full benchmark tables Bud
Appendix A.4

Long-sequence throughput & references.

Intel Xeon Platinum 8272CL — long-sequence characterization
Seq lenConc.p50p99req/s
51210025.1744.162,704
5125,0001,027.031,867.632,674
51210,0001,910.903,664.112,749
8,192100151.07262.75437
8,19210,0009,935.2119,434.16508
16,384100254.73571.22235
16,38410,00017,989.4734,860.60283
65,536100559.981,214.18100
65,53610,00050,874.3797,428.30101

References

  1. A. Vaswani et al. Attention is all you need. NeurIPS, 2017. arXiv:1706.03762.
  2. S. Wang et al. Linformer: Self-attention with linear complexity. arXiv:2006.04768, 2020.
  3. K. Choromanski et al. Rethinking attention with Performers. ICLR, 2021. arXiv:2009.14794.
  4. I. Beltagy, M. E. Peters, A. Cohan. Longformer: The long-document transformer. arXiv:2004.05150, 2020.
  5. H. Inan et al. Llama Guard: LLM-based input-output safeguard. arXiv:2312.06674, 2023.
  6. P. Chao et al. JailbreakBench: An open robustness benchmark. NeurIPS D&B, 2024. arXiv:2404.01318.
  7. L. Jiang et al. WildTeaming at scale. arXiv:2406.18510, 2024.
  8. H. Li, X. Liu, N. Zhang, C. Xiao. PIGuard (InjecGuard): Prompt injection guardrail. ACL, 2025. arXiv:2410.22770.
  9. Meta AI. Prompt-Guard-86M. Model card, 2024.
  10. Meta AI. Llama Prompt Guard 2 (86M & 22M). Model card, 2025.
  11. Katanemo Labs. Arch-Guard: Jailbreak detection model. Model card, 2024.
  12. ProtectAI. DeBERTa-v3-base prompt injection v2. Model card, 2024.
  13. Qualifire. Prompt Injection Benchmark. Dataset, 2024.
Bud Ecosystem · Simplifying Intelligence12