WMPI 2006, Austin, Texas © 2006 John C. Koob An Empirical Evaluation of Semiconductor File Memory as a Disk Cache John C. Koob Duncan G. Elliott Bruce F. Cockburn VLSI Design Lab ECE Department University of Alberta Edmonton, Alberta Canada
WMPI 2006, Austin, Texas © 2006 John C. Koob
An Empirical Evaluation of Semiconductor
File Memory as a Disk Cache
John C. Koob
Duncan G. Elliott
Bruce F. Cockburn
VLSI Design Lab
ECE Department
University of Alberta
Edmonton, Alberta
Canada
WMPI 2006, Austin, Texas Slide 2 John C. Koob, University of Alberta
Outline
Motivation
Extended Storage
File Memory
Experimental Platform
Cost/Performance Analysis
Conclusions
Future Work
WMPI 2006, Austin, Texas Slide 3 John C. Koob, University of Alberta
Motivation
Source: Computer Architecture, Hennessy & Patterson, 2003
WMPI 2006, Austin, Texas Slide 4 John C. Koob, University of Alberta
Access Time Gap Problem
Use Extended Storage Cheaper per bit than main memory Faster than disk Slower than main memory Not necessarily semiconductor media Potential for power savings
How to fill the access time gap?
WMPI 2006, Austin, Texas Slide 5 John C. Koob, University of Alberta
Historical Systems
Extended storage first appeared in expensive systems
IBM 3090 mainframe Main memory: 0.5 GB Extended storage: 4 GB Terminology: Expanded Storage
Image courtesy of www.ibm.com
WMPI 2006, Austin, Texas Slide 6 John C. Koob, University of Alberta
Historical Systems
Extended storage first appeared in expensive systems
Cray Y-MP supercomputer Main memory: 1 GB of 15-ns bipolar SRAM Extended storage: 4 GB of 50-ns DRAM Terminology: Solid State Disk
Image courtesy of the Charles Babbage Institute
WMPI 2006, Austin, Texas Slide 7 John C. Koob, University of Alberta
Recent Research
Compressed caching (1999-2003) Use compression to reduce paging costs Adaptive sizing of the compressed portion required Evaluated using a modified Linux 2.4 kernel
Multi-level main memory (WMPI 2004) 30% of memory must run at DRAM speed Remaining memory can be slower
Hardware compression Remote memory box Portions can be powered down
WMPI 2006, Austin, Texas Slide 8 John C. Koob, University of Alberta
Extended Storage Today?
Emerging technology may prompt
a return to extended storage
Semiconductor file memory Up to 5 times slower than DRAM Must be cheaper per bit than DRAM
MEMS probe-based storage 5 times faster than disk 10 times more expensive than disk
WMPI 2006, Austin, Texas Slide 9 John C. Koob, University of Alberta
What is File Memory?
File memory leverages current DRAM technology
DRAM design constraints increase costs per bit 100% of nominal capacity must be functional Contiguous address space Consistently good access time
File memory relaxes DRAM’s design constraints Bad block marking to improve yield Address space is not contiguous Improve density at the expense of performance
(e.g. multi-level DRAM or hardware compression)
WMPI 2006, Austin, Texas Slide 10 John C. Koob, University of Alberta
Feasibility of File Memory
A precedent for file memory exists
in the non-volatile memory market
NOR Flash memory Limited capacity Moderate reliability Random-access supported
NAND Flash memory High capacity Low reliability bad block marking Restricted to sequential access
Contiguous Memory
Non-Contiguous Memory
WMPI 2006, Austin, Texas Slide 11 John C. Koob, University of Alberta
Extended Storage Disk Cache
To evaluate file memory as extended storage: Require an experimental platform Modify Linux 2.4.18 OS kernel
ESDC Design Summary High memory support Page cache containment Configurable performance CPU caching issues Performance metrics Verification
WMPI 2006, Austin, Texas Slide 12 John C. Koob, University of Alberta
Configurable Performance
Need configurable file memory properties Capacity Access time
How to model different file memory access times? Use multiple page copies Gives accurate file memory slowdown ratios Problem:
Repeated page copies would be cached
Solution: Use IA-32 memory type range registers (MTRRs) Disable CPU caches for ESDC
WMPI 2006, Austin, Texas Slide 13 John C. Koob, University of Alberta
Experimental Setup
Experimental Platform Processor 2.4 GHz Pentium 4 Memory 2 GB DDR SDRAM Hard disk 18-GB Seagate SCSI Disk buffer 4-MB
Experimental Suite PostMark – benchmark for many small files Bonnie – file system benchmark Kernel compilation – Linux kernel build
WMPI 2006, Austin, Texas Slide 14 John C. Koob, University of Alberta
Postmark Results for Original Hierarchy
WMPI 2006, Austin, Texas Slide 15 John C. Koob, University of Alberta
Postmark Results using File Memory
WMPI 2006, Austin, Texas Slide 16 John C. Koob, University of Alberta
Postmark Results Analysis
Need 39% more file memory for equivalent performance
WMPI 2006, Austin, Texas Slide 17 John C. Koob, University of Alberta
Summary of Postmark Results
0 256 512 768 1024
Additional Memory Capacity (MB)
2
3
4
5
File
Me
mo
ry S
low
do
wn
Equivalent Performance Comparison
File Memory
DRAM
WMPI 2006, Austin, Texas Slide 18 John C. Koob, University of Alberta
Conclusions
Use non-contiguous file memory for extended storage Leverage DRAM cell technology Relax DRAM design constraints Use bad block marking
Preliminary evaluation of ESDC Use file memory up to 4 times slower than DRAM Even though ESDC replaces the page cache,
system performance can be improved
Ongoing research Evaluate hierarchies with file memory and page cache
WMPI 2006, Austin, Texas Slide 19 John C. Koob, University of Alberta
Selected References
Bray. Bonnie. www.textuality.com/bonnie, 1996.Castro et al. Adaptive compressed caching. Symp. on Comp. Arch.
And High Performance Computing, Nov. 2003.Ekman and Stenstrom. A case for multi-level main memory. WMPI
2004.Hennessy and Patterson. Computer architecture: A quantitative
approach. Third Edition, 2003.Katcher. PostMark: A new filesystem benchmark. TR3022, Network
Appliance, Oct. 1997.Koob et al. Test and characterization of a variable capacity multilevel
DRAM. In Proc. VLSI Test Symp., pp. 189-197, May 2005.Uysal et al. Using MEMS-based storage in disk arrays. FAST 2003,
pp. 89-101.