Research context

Fleet Sim is a fleet-planning tool. It borrows familiar abstractions from queueing, inference simulation, heterogeneous serving, and disaggregated prefill/decode research, but it does not reproduce any one research system.

This page helps choose the right level of tool. It deliberately avoids copying paper benchmark numbers into product guidance; performance claims are tied to each paper's workload and evaluation environment.

Where Fleet Sim fits

Layer	Primary question	Fleet Sim coverage
Serving engine	How should one replica batch and schedule tokens?	Represented through a calibrated profile, not simulated at kernel fidelity
Replica configuration	Which tensor/pipeline parallel and runtime settings should one replica use?	Input assumption; ComputedProfile can explore rough sensitivity
Fleet planning	How many pool instances are needed and how should traffic be split?	Primary scope
Runtime control	When should a live fleet scale, spill traffic, or reduce load?	Can evaluate static scenarios; does not operate the controller
Facility energy	What is the whole-system power and grid impact?	GPU board-power estimate only

Use a profiler or high-fidelity engine simulator to calibrate a replica, Fleet Sim to compare fleet topologies, and a production load test to accept the result.

Adjacent research

Heterogeneous fleet selection

Mélange studies cost-aware selection of GPU types across workload slices. Its core question is which measured hardware profiles to combine. Fleet Sim focuses on pool counts, routing, and queueing once performance profiles and costs have been supplied.

The common lesson is that a GPU SKU cannot be ranked without workload size, arrival rate, SLO, model, and price. Fleet Sim's built-in profile names should therefore be treated as inputs to replace, not a universal hardware ranking.

Per-replica and engine simulation

Vidur uses profiled operation-level models to simulate an LLM serving engine and search its configuration. This is a higher-fidelity layer than Fleet Sim's W/H request-level profile.

AIConfigurator explores model and engine configuration using hardware- and operation-level performance information. Fleet Sim's computed profile has a roofline-style decomposition, but that does not make it a validated substitute for AIConfigurator or a kernel database.

Measurements or selected settings from this class of tool can be converted into a ManualProfile before fleet sizing.

Disaggregated prefill and decode

DistServe and Splitwise study systems that separate the prefill and decode phases. They model or measure details such as phase interference, parallel configuration, placement, and KV transfer.

Fleet Sim's disagg command is much narrower: it applies fixed phase degradation and TTFT correction factors while sweeping prefill and decode worker counts. It is useful for a first sensitivity study, not for validating network topology or KV-transfer latency.

Autoscaling and burst control

SageServe considers forecast-aware runtime capacity control, while TokenScale considers runtime scaling for disaggregated inference using stage-level token demand.

Fleet Sim does not implement either control loop. Its whatif output can help identify static rates and fleet shapes that a separate controller should test, but it does not model startup time, forecast error, control delay, or live backpressure.

Power-aware inference

Power-aware serving work, including GPU-to-Grid, motivates studying the latency effect of concurrency caps. Fleet Sim's grid-flex command represents this as a fitted or estimated batch-versus-board-power curve plus analytical and optional DES latency.

It does not measure facility power or implement demand response. See the power model for calibration and reporting boundaries.

Queueing foundation

Analytical sizing uses Erlang-C waiting probability and a Kimura-style M/G/c tail approximation. The DES then evaluates selected candidates with explicit synthetic arrivals and KV-slot admission.

These models answer a different question from kernel or serving-engine simulation: they estimate fleet queueing once a service-time distribution is known. Their assumptions—stationary Poisson arrivals, approximate independent service times, and logical KV slots—must be checked against the deployment.

Selecting a tool

If you need to decide...	Start with...
batching, scheduler, or parallel settings for one replica	a serving-engine benchmark or high-fidelity simulator
which measured GPU/model profile is cheapest	heterogeneous configuration search
short/long pool counts and routing sensitivity	Fleet Sim optimize, pareto, and simulate
prefill/decode worker-count sensitivity	Fleet Sim disagg, followed by a disaggregated-system test
second-by-second scaling behavior	a runtime autoscaling/controller model
board-power sensitivity to concurrency	Fleet Sim power commands after measurement
production capacity approval	a production-shaped load test

No tool in this table removes the need to preserve its workload, model, hardware, runtime, and measurement assumptions with the result.