If your GPUs already support hardware partitioning, why add a software layer on top? It is a fair question, and the honest answer is that the two solve overlapping problems with very different constraints — and most fleets contain more cards that cannot do MIG than cards that can.
Multi-Instance GPU (MIG) is a hardware feature introduced with the Ampere architecture and continued in Hopper. It partitions a single physical GPU into up to seven isolated instances, each with dedicated memory, streaming multiprocessors, and memory controllers.
FCSP (Fixed Capacity Spatial Partition) is a software GPU virtualization layer that achieves similar multi-tenant isolation through LD_PRELOAD interception, token-bucket rate limiting, memory quota enforcement, work-conserving scheduling, and intelligent prefetching. The architecture is covered in depth in the FCSP framework post.
Where MIG runs out of road
- Hardware requirements. MIG support is limited to A100, A30, H100 and H200. Consumer GPUs and older data centre accelerators like V100 or T4 do not support it at all.
- Static partitioning. Partitions are fixed at creation. Changing them requires no running CUDA processes, MIG mode enabled, and a GPU reset.
- Limited partition profiles. Only predefined profiles exist — 1g.10gb through 7g.80gb — so you take the sizes on offer rather than the sizes your workloads need.
- Wasted resources. With static allocation, idle instances cannot share. When one tenant of three is active, that tenant stays capped at 33% while 67% of the card sits idle.
How FCSP answers each one
- Universal GPU support. All CUDA-enabled GPUs, from data centre (A100, H100, V100, T4, A10, L4) through professional and consumer cards down to legacy parts — minimum CUDA Compute Capability 3.0.
- Dynamic allocation. Partitions adjust at runtime without disrupting workloads, driven by environment variables such as
BUD_SM_LIMITandBUD_MEMORY_LIMIT. - Arbitrary splits. Any percentage. One tenant at 25% compute and 4GB, another at 35% and 6GB, a third at 40% and 10GB — not a profile from a fixed menu.
- Work-conserving scheduling. Idle capacity flows automatically to active tenants. Benchmarks show efficiency up to 143.9% relative to static allocation.
Architecture
The core is libvgpu.so, which orchestrates resource allocation across a memory manager, a compute throttler, a stream classifier, NCCL hooks, UVM prefetching, and graph optimisation. An SM observer thread continuously monitors GPU status via NVML, calculates fair-share allocations, and drives dynamic rebalancing.
Four isolation modes
- None. No isolation, minimal overhead — around 40ns per API call.
- Balanced. The default: a 20% floor, a 40% shared pool, and a 40% burst pool.
- Strict. Hard quotas, no bursting — MIG-like behaviour in software.
- Adaptive. Switches automatically based on observed contention.
Rate limiting uses a token bucket with per-stream buckets, batch token consumption that cuts atomic operations by 8–16×, a PID controller, and exponential backoff. Memory management supports absolute limits, percentage allocation, hard and soft enforcement, and per-device limits; UVM adds prefetching, pressure monitoring, automatic eviction (LRU, access-aware, or FIFO) and oversubscription.
Benchmarks
Measured on an RTX 3080 (10GB, 68 SMs) with FCSP in adaptive isolation over 100 iterations. MIG figures are for an A100, since the 3080 cannot run MIG at all — which is itself the point.
| Metric | Native | FCSP | MIG (A100) |
|---|---|---|---|
| Kernel launch latency | ~3 µs | 4.9 µs | ~3.5 µs |
| Memory alloc latency | ~100 µs | 704 µs | ~100 µs |
| API interception | 0 ns | 40 ns | 0 ns |
| Rate limiter | — | 1.5 µs | — |
| Metric | FCSP balanced | FCSP adaptive | MIG |
|---|---|---|---|
| Fairness index | 0.996 | 0.996 | 1.0 |
| QoS consistency (CV) | 0.07 | 0.07 | <0.05 |
| Noisy-neighbour impact | 11.5% | 4.66% | ~3% |
| Cross-tenant isolation | 81.4% | 81.4% | ~95% |
| Metric | FCSP | MIG |
|---|---|---|
| Affinity complementary efficiency | 143.9% | — |
| Work-conservation benefit | up to 67% | 0% |
| Utilisation, 3 idle / 1 active | ~100% | 33% |
The isolation table is the honest one: MIG wins on every isolation metric. FCSP reaches 81.4% cross-tenant isolation against MIG's ~95%, and its QoS consistency is measurably looser. The efficiency table is where that trade is repaid.
Feature comparison
| Feature | FCSP | MIG |
|---|---|---|
| Consumer GPU support | Yes | No |
| V100, T4, A10 support | Yes | No |
| A100, H100 support | Yes | Yes |
| Arbitrary splits | Yes | No — fixed profiles |
| Dynamic resizing | Yes | No |
| Changes without disruption | Yes | No — requires GPU reset |
| Work conservation | Yes | No |
| Burst capability | Yes | No |
| Memory isolation | Software | Hardware |
| Compute isolation | Software | Hardware |
Which to choose
- You do not have MIG-capable GPUs — on consumer or older data centre cards it is the only multi-tenant option.
- You need runtime resizing without disrupting workloads, as in Kubernetes clusters where jobs need variable resources.
- Workload patterns vary — dev environments with burst testing, staggered batch processing, time-sharing.
- You can co-schedule complementary work: pairing compute-heavy with memory-heavy workloads reaches 143.9% of isolated efficiency.
- You are serving LLMs, with bursty allocation, mixed compute patterns and variable batch sizes.
- Maximum isolation is critical — safety-critical work such as financial trading or medical imaging benefits from hardware fault isolation.
- A regulatory framework requires hardware partitioning specifically.
- You need hard QoS guarantees; FCSP carries software scheduling variance of 0.07 CV.
- You have A100s or H100s running static, predictable, always-on workloads — MIG is simpler to operate with zero runtime overhead.
A 30-minute walkthrough on your hardware mix, governance constraints, and top use case.
Getting started
# build cd bud_fcsp && mkdir build && cd build cmake .. -DCMAKE_BUILD_TYPE=Release make -j8 # run any CUDA workload under FCSP LD_PRELOAD=/path/to/libvgpu.so python train.py
For Kubernetes, FCSP settings come from a ConfigMap with pod-level isolation set through BUD_ISOLATION_MODE, BUD_SM_LIMIT and BUD_MEMORY_LIMIT. Three profiles cover most deployments: a development cluster maximising flexibility (adaptive, 10% floor, 30% shared, 60% burst); production ML serving balancing isolation against efficiency (balanced, 40% floor, 30% shared, 30% burst); and high-isolation multi-tenant (strict, 100% floor, no sharing or bursting).
Tuning
To reduce overhead: keep fast paths enabled, raise the batch token size from 16 to 64 for high throughput, and switch off metrics and verbose logging in production. FCSP uses AVX2 and SSE4.2 automatically where available.
To improve isolation: use adaptive mode for dynamic workloads, raise the floor percentage for stronger guarantees, enable the PID controller for smoother throttling, and drop the observer interval from 5ms to 2ms for faster response.
Limitations, both ways
- Software isolation only — vulnerable to a genuinely malicious tenant in a way hardware enforcement is not.
- Roughly 2µs per kernel launch and 600µs per memory allocation in overhead.
- No hardware error isolation: a GPU fault affects every tenant.
- Requires
LD_PRELOAD, so the application must use dynamically-linked CUDA. - Metrics are approximated — SM utilisation is polled at a 5ms interval.
- Four supported GPU families, and nothing else.
- Fixed profiles — arbitrary partition sizes are not possible.
- Static allocation: resizing requires terminating the workload.
- No work conservation, so idle capacity is simply wasted.
- The sum of MIG instances delivers less than the full GPU.
Conclusion
The choice between FCSP and MIG is not about which is better — it is about which fits the constraint you actually have. For most scenarios FCSP provides 80–95% of MIG's isolation benefit with significantly more flexibility and universal hardware support, and its work-conservation improves cluster utilisation by 40–67% in realistic multi-tenant conditions.
Where hardware-enforced isolation is a compliance requirement rather than a preference, MIG remains the right answer — on the four GPU families that offer it.
- MIG is stronger on every isolation metric; FCSP repays that on utilisation and flexibility.
- Work conservation is the structural difference — MIG strands idle capacity by design.
- Most fleets contain more cards that cannot run MIG than cards that can.
FCSP figures are measured on an RTX 3080 (10GB, 68 SMs) in adaptive isolation over 100 iterations. MIG figures are for an A100 and are therefore a cross-hardware comparison, not a like-for-like one — the 3080 cannot run MIG. Efficiency figures above 100% describe complementary co-scheduling against isolated execution of the same pair of workloads. Overhead and isolation results will vary with GPU, driver, tenant count and workload mix; reproduce with GPU-Virt-Bench on your own hardware before sizing capacity against them.
