Wednesday, March 25, 2015
Disk Access Faster than RAM?
Talk about some sloppy research. Here's the article. It has links to the original research paper. Go read, at least the somewhat lighter article; I'll wait.
OK, so I guess it comes to a surprise to some people, but hand-packing on-the-fly-allocated storage using Java or Python probably is slower than slinging the same information to disk. Because, I don't know, since about the Carter administration, OSs and I/O controllers have stored things in (usually) dedicated ram that allowed the CPU to rampage on with better tasks. Then the relatively inexpensive controller could do the waiting for the disk to come around for the next write. This is not rocket science, this is rudimentary data buffering. Nothing I wasn't doing with a PDP-11 30 years ago. Except that in that case it was nuclear data to 9-track tapes, which were orders of magnitude slower than the processor could theoretically feed them. Even so, with clever pre-culling of the data and double-buffering schemes, the output times were essentially nil.
I notice though that the article does not address getting the data back off the disk. That is another matter entirely. But even that can have remarkable efficiencies if the the read cycles are not entirely random. Why are these techniques mysteries to the authors?
Use the tools before you, people. Some very smart engineers have spent decades sharpening them, and I doubt you'll do any better by hand-knitting your own flint blades.