vLLM tuning, from 400 to 1.2k tok/s.

Seven knobs. Most teams touch two. Here's the set we check on every install, with the numbers we saw on a 70B workload going from 400 to 1200 tok/s.

## The knobs

1. **tensor-parallel-size** — match it to your NVLink topology, not your GPU count.
2. **max-num-seqs** — the hidden one; defaults are conservative.
3. **gpu-memory-utilization** — leave 5% headroom, take the rest.
4. **swap-space** — zero is usually right. Nonzero kills tail latency.
5. **enable-prefix-caching** — free throughput if your workload has shared prompts.
6. **quantization** — AWQ vs. GPTQ vs. FP8 is a workload question.
7. **speculative decoding** — the big one, if your draft model is tuned.

## The numbers

| knob | before | after | delta |
|------|-------:|------:|------:|
| max-num-seqs | 128 | 512 | +40% |
| prefix-caching | off | on | +18% |
| tp size | 4 | 8 | +22% |
| spec decode | off | on | +65% (shared) |

Your mileage will vary. Measure with your traffic, not ours.