Preview - rendered architecture diagrams; full validated atlas + dataset coming.
Back to models

MegaBlocks dropless-MoE (dMoE block, grouped-GEMM token routing)

history megablocks dropless validation pending

Architecture diagram

Rendered with TorchLens / Graphviz

Open SVG
Unrolled - full
MegaBlocks dropless-MoE (dMoE block, grouped-GEMM token routing) architecture diagram