Lecture 5: TopDown Microarchitecture Analysis: What Is Holding the CPU Back?

An application runs slower than expected. Memory seems like a plausible explanation. But how do we establish whether memory is actually the bottleneck, and where should we look next?

In Lecture 5 of It’s the Memory, Stupid!, Harald Servat (Intel) and Victor Xirau (BSC) introduce TopDown Microarchitecture Analysis: a systematic way to investigate what limits execution inside a CPU core. The method starts with a broad classification and progressively narrows the diagnosis. Lecture slides

Modern processors expose hundreds of hardware performance events. Turning those measurements into a useful explanation can be difficult. TopDown gives the investigation a structure by accounting for pipeline slots: opportunities for the processor to handle micro-operations, the smaller operations into which instructions are decoded.

At the top level, those slots fall into four categories:

• Retiring: operations that complete and commit their results.
• Bad Speculation: resources spent on work that is discarded, or on recovering from incorrect speculation.
• Frontend Bound: the instruction fetch and decode machinery cannot supply operations quickly enough.
• Backend Bound: the execution side cannot accept more operations because resources are unavailable.

These categories help identify which direction deserves further investigation. A high Retiring fraction, however, does not establish that the software is fully optimized; opportunities such as vectorization may remain. Intel’s TopDown guide

For a course about memory, the Backend Bound branch is particularly interesting. It separates core execution constraints from memory constraints. Following the memory branch further helps locate limitations within the cache hierarchy, main memory, or store handling.

That distinction matters. “Memory bound” is the beginning of a diagnosis. If DRAM accesses are limiting performance, one useful next step is to understand the role that bandwidth and latency play in that limitation. The answer helps guide decisions about data locality, cache use, and the characteristics of the memory system. The session introduces practical routes into the methodology through BSC’s TopDown Visualizer, Intel VTune Profiler, and Linux perf. Lecture slides

This adds another perspective to the course by covering different approaches to performance analysis. Roofline models provide a view of computational and memory limits, the bandwidth–latency analysis behind Mess and PROFET focuses on memory-system performance, and TopDown examines how applications use the underlying hardware. About the series

Explore the course materials for the slides, reference papers, and further tools, or visit the recordings page to follow the series.