The roofline model reduces performance analysis to one comparison: the time a computation takes is approximately the maximum of its FLOPs divided by the chip’s compute rate and its bytes moved divided by the chip’s memory bandwidth. Plotted against arithmetic intensity, achievable performance rises along a bandwidth-limited diagonal and then flattens at the compute peak, forming the “roofline” shape.
For LLMs it is the single most useful lens on both training and serving: large-batch matrix multiplications sit on the compute-bound roof, while one-token-at-a-time decoding sits far down the memory-bound slope. The same analysis applies at every level of the stack by swapping in the relevant bandwidth (HBM, chip-to-chip links, or datacenter network).