MoonEP routing flow — live

September 3, 2026 · source: MoonshotAI/MoonEP

An interactive model of one MoonEP MoE layer. Set the shape, drag the router skew, and watch: how the router loads each expert, how the planner's greedy balance repairs the imbalance, which expert weights cross which link, and how long the routed-expert path takes — rank by rank, barrier waits included. Everything is computed by a faithful JS port of MoonEP's planning kernel (revision 2bd860b, 2026-08-13).

Routing flow who sends what to whom, and what it costs

Read top to bottom, in the order the planner works. First, which experts the router loaded, and how far each rank ends up over or under capacity. Then the greedy balance that repairs the imbalance, one move at a time. Then where the replica weights fly, next to every token src→dst pair. Finally, what it all costs in time, rank by rank — and the memory it all lives in. Every chart recomputes live from the sliders through the planner port (Steps 1–4, the C2 destination arithmetic, per-token dedup) — the traffic shown is what the kernels would actually move, not a sketch.

Per-expert breakdown

One bar per expert, colored by its owner rank (the same colors as the tables and the balance log). Router skew shows up as uneven bars; a hot rank shows up as one tall colored block. Each bar splits in two: the solid base is entries computed on their own source rank (no wire traffic), the faded top is entries sent to another rank. Hover a bar for the planner's full decision on it. The tallest bar is marked ▾: MaxVio (bench_vs_deepep.py:84) is how far it sits above the dashed average line, as a ratio − 1.

Same data, summed per rank. Add each rank's experts together and subtract its capacity (S·K entries — what its GPU will compute after balancing). Bars above zero have too many entries; bars below zero have room to spare. The greedy balance below moves exactly these surpluses into exactly these deficits.

The greedy balance, step by step

The planner's two greedy loops, replayed for this exact routing (planning.py:672–701 Step 2, :746–768 Step 3). Each move refills the currently hungriest rank in one shot, from the currently biggest donor. The donor can overshoot into deficit; it then becomes a receiver and gets refilled by a later move. That is the deliberate price of one-donor-per-receiver. Each move is realized by slicing off the donor's biggest experts, and every slice of a foreign expert ships one weight copy.

Where the weights fly

Each orange arrow is one duplicated expert's weights, flying from its owner to the rank that will compute it. Width scales with the rows the copy serves; dashed means it crosses a node boundary. Token traffic lives in the matrix beside it — every src→dst pair.

token payload rows · src ↓ dst →

entries routed to each rank's experts

by the router, before balancing — after planning every rank will receive exactly S·K

expert weight copy, owner → receiver · width ∝ rows served · dashed = cross-node (IB)

How long do one layer's routed experts take: dispatch → grouped GEMM → combine?

Just the routed-expert path of one layer — planning, dispatch, prefetch, grouped GEMM, combine (attention, the router, and the shared experts are not in these bars) — forward + backward, rank by rank, assuming no stragglers. Every rank enters each phase together, so all idle time is structural: a rank with lighter comm finishes early and stalls at the next cross-rank barrier until the busiest rank arrives. Same bandwidth and TFLOP sliders as everything else; hover any segment.

forward, critical path

backward (excl. reduce_grad)

routed-expert total (fwd + bwd)

avg barrier wait per rank

planning + sync token comm (dispatch / combine) weight comm (prefetch; faded tail = reduce_grad, overlaps next layer) local dedup kernels grouped GEMM waiting at a barrier

A bandwidth-only model. Each comm segment is that rank's max(egress, ingress) ÷ link bandwidth; GEMM is FLOPs ÷ effective TFLOP/s. The throughput slider is effective, so dtype and utilization fold into one number: H100 peaks ≈989 TFLOP/s dense BF16 and ≈1979 FP8; B200 roughly 2.2×/4.5× that; a grouped GEMM over ragged expert segments typically lands at 40–60% of peak. The default 400 ≈ H100 BF16 at ~40%. Token rows are always bf16 (H×2 B — MoonEP has no fp8 dispatch). Weight-copy bytes follow the precision toggle: train = 3·H·H′×2 B bf16 plus a 2× fp32 reduce_grad return; infer = K3's MXFP4, packed 4-bit + one scale per 32 elements ≈ 3·H·H′×0.53 B, with no backward pass at all. Barrier hardware latency (~µs each) is not modeled, so the gray is purely structural waiting. With node size < R, ranks form NVLink islands and every cross-node edge is priced at the IB bandwidth; intra- and inter-node transfers ride separate links, in parallel. Note that the multi-node topology is itself a what-if: MoonEP requires an NVLink/NVSwitch domain (VMM symmetric memory + multimem). Finally, the comm blocks need not be dead time: the K3 technical report overlaps this EP communication with the two shared experts each layer. Their GEMMs don't depend on routing, so they slot into exactly these comm windows — the timeline here shows the unoverlapped critical path.

Memory: reserved once, breathing never

One rank's memory over the same time axis as the timeline above. Every MoonEP band is reserved at t = 0, before the router routes a single token — the light fill is the static reservation, the solid fill is what's actually occupied as the layer runs. The only memory that appears and disappears is the framework's own GEMM activations. That flatness is the point: fully static shapes, no per-layer allocation, no fragmentation, no OOM at any imbalance (README's comparison vs dynamic EP). Hover any band for its formula.