PD-Disaggregated Inference Deployment with sgl-project/rbg
Use RoleBasedGroup to run SGLang Prefill/Decode disaggregation as one operable unit: routing, startup dependencies, multi-node tensor parallelism, KV transfer, and coordinated rollouts.
Category archive
GPU Acceleration
4 practical articles on GPU / GPU Acceleration, covering setup, troubleshooting, and production decisions.
More topics
Other topics
Selected pieces
Featured in this category
KAI-Scheduler vs HAMi: Scheduling Accounting and Runtime Enforcement
Compare KAI-Scheduler Reservation Pod accounting with HAMi CUDA API-level memory and compute enforcement, including their boundary versus hardware isolation such as MIG.
hetGPU: What Cross-Vendor GPU Binary Portability Requires
A review of the hetGPU research prototype that separates paper design, conceptual pseudocode, and current cross-vendor GPU engineering capability.
gpu-manager Architecture: Device Plugin, vCUDA, and Migration
Review tkestack/gpu-manager device registration, vCUDA resources, library injection, topology allocation, CUDA limits, and migration choices.
Archive
All posts in this category
All posts in reverse chronological order.
PD-Disaggregated Inference Deployment with sgl-project/rbg
Use RoleBasedGroup to run SGLang Prefill/Decode disaggregation as one operable unit: routing, startup dependencies, multi-node tensor parallelism, KV transfer, and coordinated rollouts.
KAI-Scheduler vs HAMi: Scheduling Accounting and Runtime Enforcement
Compare KAI-Scheduler Reservation Pod accounting with HAMi CUDA API-level memory and compute enforcement, including their boundary versus hardware isolation such as MIG.
hetGPU: What Cross-Vendor GPU Binary Portability Requires
A review of the hetGPU research prototype that separates paper design, conceptual pseudocode, and current cross-vendor GPU engineering capability.
gpu-manager Architecture: Device Plugin, vCUDA, and Migration
Review tkestack/gpu-manager device registration, vCUDA resources, library injection, topology allocation, CUDA limits, and migration choices.