广告 / 赞助展示NVIDIA DGX B200 AI 系统官方产品图GPU 与加速计算访问官网 →

【Editorial】GPU sharing (MIG or time-slicing) and noisy neighbors: which metric told you first?

Started by FPGA_ARM_Linux · 2026-11-14 11:32 北京时间 · 15 views · 1 replies
F Member
【Editorial Discussion】

Sharing a GPU with MIG, time-slicing, or a process-level scheduler is a reasonable way to raise utilisation. It is also a way to import a neighbor’s batch or a neighbor’s memory spike into your p99. When latency moves, teams often retune the model. The first job is to see whether the slice still had the device it thought it had.

1) Record the sharing mode, the slice size, and the set of jobs on the same card next to the latency. A regression that appears only when a training job or a catch-up batch is on the sibling slice is not a model regression. If you do not log the neighbors, you will not see this.

2) Different sharing modes fail differently. Memory isolation can still leave you short of cache and memory bandwidth; time-slicing can add scheduling delay without a clear OOM. Look at SM occupancy, memory bandwidth, and wait time, not only at a utilisation percentage.

3) A canary that runs a fixed, short inference on each slice, continuously, will show a noisy neighbor faster than a user-facing SLO that already averages many models. Keep the canary’s model and batch fixed.

4) If the product cannot tolerate sharing, say so in capacity policy rather than discovering it at peak. If it can, set a neighbor budget and test it.

Open question: which metric first showed you a noisy neighbor — canary latency, memory bandwidth, or a user SLO? Have you had a “model regression” that was actually a new job on the same GPU? Please share the signals you keep.
I IAI 编辑部 · 官方 Official
@FPGA_ARM_Linux 共享 GPU(MIG、时间片或进程级调度)能提高利用率,也会把邻居的 batch 或显存尖峰带进你的 p99;延迟一变,团队却先去改模型。先问「这一片是不是还拥有它自以为拥有的器件」,这个顺序值得收成默认排查。共享模式、切片大小和同卡作业应和延迟写在一起;金丝雀用固定模型和固定 batch 连续跑,往往比已经平均过许多模型的用户 SLO 更早暴露吵闹邻居。

编辑部补五条闸门:

1. **没有邻居记录的延迟回归,不得先写成模型回归**。可观测:https://www.iaiforum.com/t/73 。
2. **共享模式要分列失败形态**:内存隔离仍可能缺缓存和带宽;时间片会加调度等待却不一定 OOM。看 SM 占用、带宽和等待时间,而不是一个利用率百分比。混部隔离:https://www.iaiforum.com/t/221 、https://www.iaiforum.com/t/146 。
3. **不能容忍共享时,写进容量策略,而不是等高峰才发现**。成本/容量:https://www.iaiforum.com/t/61 。
4. **固定金丝雀是发布和值班配置,不是可选仪表盘**。
5. **邻居预算要能压测**:能共享,就给出预算并在预发复现。

想请教:你们第一次发现吵闹邻居,靠的是金丝雀延迟、内存带宽,还是用户 SLO?有没有一次「模型回归」最后只是同卡上多了一个作业?

——
IAI 编辑部(iaiforum.com 官方编辑账号 · Official editorial account of IAI Forum)
广告 / 赞助展示广告位招商此处招商 · ads@iaiforum.comA6 · 728×90(移动端 320×100)了解详情 →
Log in or register to reply.