For a change I want to talk about my home. Well not really, it is my attempt to explain memory (volatile and persistent) hierarchy in a computer.
Have you ever observed a kitchen (or dressing room) in your home? Let us take a look at it to understand and articulate.
1) Small Shelve (< 10 jars) very near to stove – We usually keep tea power, sugar, salt, popu (పోపు), chilly power, oil in this shelve
2) Medium Shelve (10 jars to 50 jars) in near reach of our hands, but little away from stove – We usually keep all other raw materials such as Red gram dall, Black gram dall, Idli Rawa, Bombay Rawa, Wheat Rawa, Vermicelli, etc. and utensils such as plates, glasses, utensils, etc. in this shelve
3) Big shelve (storage for cartons, gunny bags, drums, etc.) that are beyond reach of our hands and require ladder – We usually keep storage items such as rice bags, oil carton, unused utensils or rarely used items (dinner sets), etc. in this shelve
4) Apart from the shelves, we also having work area near the stove where items are brought for a temporary purpose and are placed back in their respective places when we are done with them.
I hope you now get the rough idea of how kitchen is organized. Here are the top level reasoning of organization of items.
- The time order to bring items near to stove is: Small Shelve, Medium Shelve, Big Shelve
- The more frequently you use an item, the near it is brought to the stove – Salt, Chilly Power, etc.
- The bigger the item – we split into parts and keep one part handy to use and keep remaining parts farther away – Rice bags are split into small portion (10 KG) and rest. Small portion is kept in reach and the rest can be kept far
Let us imagine a kitchen, where things are not organized by above pivoting rules/guidelines, what would be side effect? We will end-up taking more time to prepare the food than in the current model of organization.
Now let us analyze stove. Let us say, I have a stove ‘A’ that can cook a curry in an 30 minutes if all items are supplied w/o any delay (ideal case). Because there will be delay to bring the required ingredients and mix with curry, we need to count that. Here we might be lowering the stove flame to account for ingredient transport. Let us say, we spend 30 minutes in getting the ingredients for ready use. So, we can prepare one curry with stove ‘A’ in one hour. I became rich and can afford a better stove ‘B’ that can prepare a curry in just 20 minutes (in ideal case). That is, actual curry preparation time would be 50 minutes (with transport time of 30 minutes). Even richer me, better stove ‘C’ with 10 min. preparation time (ideal) – it would take 40 minutes in total.
Ouch, we are severely inefficient in bringing the ingredients near the stove. Hey, let us say you get an assistant to help. You seek help of an assistant and reduce the ingredient transport time to 20 minutes (from 30 minutes). So, with stove ‘C’, assistant help, we will be able to finish curry in 30 minutes. Wow, yummy curry in 30 minutes!
Well, we become smart at doing things upon practice. So, we learn which ingredients are required at what time, so why wait till that time? That is, we could predict the ingredients and time map, keep them ready at the right time, so that we make sure we don’t unnecessarily occupy the place near the stove. With this prediction, let us say we reduce ingredient transport to 10 minutes. That means, yummy curry preparation with stove ‘C’ and prediction is just 20 minutes!
Well, I hope with that gyan of home, kitchen, shelves, stoves, etc. Let us compare this whole system with computer and esp. memory hierarchy.
1) Small Shelves – On-Chip Processor Cache
2) Medium Shelves – RAM
3) Big Shelves – Hard Disk
4) Work area – Registers (AX, BX, CX, DX, AC, IP, etc.) + Processor Cache
5) Stove – CPU
What shall we keep in processor cache? – Most frequently accessed data. With ‘prediction’ smartness, we keep ‘next required’ data.
What shall we keep in RAM? – What we use periodically, not super frequently, but moderately required such as currently running program’s required code pages, usual program runtime data, etc.
What shall we keep on Disk? – Big files, videos, binaries, etc. If we need any file, we don’t load whole file into RAM but get parts of it (much like we get 6-10 KG of Rice from rice bag)
As we have found in the discussion, the more we spend time in ingredient transport the more we need to use stove at lower flames. That means, the more time it takes to get data from RAM/Disk, the less efficient use of CPU. Performance experts refer to it as ‘memory wall’.
Like we become smart in predicting which ingredients are required at what time to avoid using stove with lower flames in cooking, modern processors have techniques like branch prediction to reduce memory wall problem for CPU processing.
If we have 200 KGs of rice (4 Rice bags of 50 KG each), we might keep 3 full rice bags in big shelve and have another shelve where we could keep ‘currently’ used rice bag which is farther away than medium shelve but does not require a ladder. Some times, we can have kitchen w/o big shelve at all in which case we don’t have much storage space and so we wont keep that many bags of rice.
As much as we can add more types of shelves, similar stuff is applicable to computers. We now have Solid State Drives, that come between RAM and Hard Disk. We can have a computer w/o Hard Disk, it just means that we don’t have much storage space.
With all this kitchen discussion, I am feeling hungry. Will be back with another article on technology in simple English. Till then…
Laxmi Narsimha Rao Oruganti