Home/Products/Bud LayerZero
Bud LayerZero · Layer 01 · Kernel Orchestration

One API that picks the fastest kernel. Every time.

Bud LayerZero picks the fastest GPU kernel for every operation — automatically, explainably, and in nanoseconds. FlashAttention, FlashInfer, xFormers, Liger, Torch: chosen for you. Open source under Apache 2.0, PyTorch native, zero required dependencies.

Bud LayerZero Product Brief View on GitHub
Overview

The layer beneath your model.

LayerZero is a production-grade kernel orchestration framework for LLM inference. It gives you a single API across every GPU-optimised kernel library — and decides which one runs, per operation, per shape, per device. Call lz.attention() and the right kernel executes. No integration branches, no hand-tuned dispatch tables, no guessing.

<100ns selection on a cache hit <500ns on a cache miss 0ns in static dispatch mode 2,000+ unit tests

Framework design targets, backed by an MVCC sharded selection cache (256 shards, O(1) invalidation). Every selection is explainable via lz.explain().

  • Bud LayerZero is Layer 01 of the eight-layer Bud stack — the Kernel Orchestration layer, between your model and the metal. One API across FlashAttention, FlashInfer, xFormers, Liger and Torch; selection in under 100 nanoseconds on a cache hit and 0 nanoseconds in static dispatch; every decision explainable — lz.explain() shows candidates and scores; open source under Apache 2.0, PyTorch native, zero required dependencies.
  • The selection engine: a call to lz.attention(q, k, v) runs six steps — policy check (a LOCK rule wins outright), cache lookup (under 100ns on a hit), filter (hardware, dtype, shape), apply policy (allow and deny), score (priority, boosts, history), select (then cached, O(1) invalidation). The candidates race: flashinfer.decode scores 0.94 and is selected; flash_attn 0.81 eligible; xformers 0.62 eligible; torch_sdpa 0.40 fallback; liger filtered — not an attention backend. A cache miss resolves in under 500 nanoseconds; the next identical call hits the cache in under 100.
  • Six capabilities inside the layer: one API across every backend with unified dispatch; hardware-aware automatic selection that never fails closed; every decision explainable via lz.explain() and lz.which(); always overridable with lock, allow, deny and boost rules in code or hot-reloading YAML; orchestration that disappears — 0ns static, under 500ns dynamic; and built for the fleet — circuit breakers under 50ns, CUDA-graph safe, Prometheus and OpenTelemetry, 2,000+ tests.
  • The result: the fastest kernel, not a black box.
Value proposition

Kernel fragmentation, solved once.

Kernel libraries multiply faster than anyone can integrate them, and every serving framework rebuilds the same selection logic — picking differently for identical inputs. LayerZero makes that decision once, makes it visible, and makes it cost effectively nothing: call lz.attention() and the fastest available kernel executes.

Selection · cache hit<100ns
Selection · cache miss<500ns
Static dispatch0ns
Unit tests2,000+
design targets for the selection path, not kernel runtime · basis in the product brief
Key features

What a dispatch table can't give you.

Most frameworks pick a kernel and never tell you why. These six are why the selection layer is worth having — and why it never shows up in your latency budget.

01

One API across every backend

FlashAttention, FlashInfer, xFormers, Liger, Torch SDPA and more behind one interface — add a library through a standardised adapter and every caller benefits.

5+ backendsone interface · no integration branches
02

Hardware-aware, automatic

Selection accounts for the device, dtype, and shape in front of it — and degrades gracefully to torch SDPA, so it never fails closed.

4 hardware targetsCUDA · ROCm · Intel XPU · CPU
03

Every decision explainable

lz.explain() returns the candidates, their scores, and what filtered them out; lz.which() shows what would run before it runs.

0 silent choicescandidates · scores · filter reasons
04

Policy without a redeploy

Lock, allow, deny, and boost rules in strict priority order — in code or YAML that hot-reloads, so you can A/B a kernel on a live fleet.

4 rule typeshot-reloaded · 0s downtime
05

Orchestration that disappears

Static dispatch adds literally nothing, and a warm MVCC cache answers repeat selections in under 100 nanoseconds — flexibility with no latency tax.

0ns static<100ns hit · <500ns miss · 256 shards
06

Built for the fleet

Circuit breakers checked in under 50ns, CUDA-graph capture safety, readiness checks with JIT warmup, and Prometheus and OpenTelemetry built in.

2,000+ testsproperty-based · fuzz · stress · correctness
Go deeper

The full story, in depth.

The six-step selection pipeline, the full backend and operation coverage, the policy system, and the performance targets with what backs them.

Product brief

Product Brief

Bud LayerZero Product Brief

The deep-dive product reference

The selection pipeline step by step, every backend adapter, the four-tier policy system with worked YAML, dispatch modes, and the production hardening in full.

Read the product brief
Open source

GitHub

LayerZero on GitHub

Apache 2.0 · PyTorch native

Bring your stack, keep your kernels, and let the selection layer disappear into the background. Zero required dependencies — every backend optional.

View the repository
Get started with Bud

Put your data on it.

The fastest way to see what an integrated AI operating system does for your enterprise is a proof-of-concept on your infrastructure, with your data.

Request a POC Talk to the team
01 Identify a use case where complexity, cost, or governance is a known pain point.
02 Joint discovery — Bud maps your AI pain points to platform capabilities.
03 POC in days, on your hardware, with your data.