Lecture 1: Let's Challenge a Few Assumptions
We challenge a common line of reasoning: DRAM latency has remained almost constant for decades, so there is little we can do about it.
The introductory lecture lets the audience experience the memory wall and discusses how it has evolved over the last three decades. We also challenge a common line of reasoning: DRAM latency has remained almost constant for decades, so there is little we can do about it.
Our objective is to challenge this learned helplessness. There are many things users can do with the systems we have today, but the first step is understanding what is really happening.
The Latency That Matters Is Not Just DRAM Latency
From the application’s perspective, what matters is load-to-use latency: the time between issuing a memory operation and having the requested data available to the processor.
That latency is determined by much more than the intrinsic delay of the DRAM cells.
And that is good news.
Once we recognize that the real bottleneck is not simply in the DRAM cell, we can start doing something about it.
Bandwidth Is Equally Easy to Misunderstand
Memory bandwidth creates similar misconceptions.
Imagine a system advertised as providing 100 GB/s of memory bandwidth, while an application is using 70 GB/s. Would increasing the available bandwidth improve performance?
At first glance, perhaps not: 70 is comfortably below 100.
But, believe it or not, memory bandwidth can still be the main bottleneck directly limiting application performance in this situation.
Why?
That is exactly the kind of question we address during the course. Data-sheet bandwidth, achievable system bandwidth, application bandwidth consumption, latency, and contention are tightly interconnected concepts. Understanding how they interact is essential for identifying the real performance bottleneck.