C++实现的IO高效的算法
类似的有berkeley DB Hash等
TPIE
The data sets involved in some modern applications are too large to fit in the main memory of even the
most powerful computers and must therefore reside on disk. Thus communication between internal and
external memory, and not actual computation time, often becomes the bottleneck in the computation.
This is due to the huge difference in access time of fast internal memory and slower external memory such
as disks. While typical access time of main memory is measured in nanoseconds, a typical access time of
a disk is on the order of milliseconds [20]. So roughly speaking there is a factor of a million difference in
the access time of internal and external memory. A good example of an applications involving massive
amounts of geometric data is NASA’s Earth Observation System (EOS) [29, 42], which is expected to
manipulate petabytes (thousands of terabytes, or millions of gigabytes) of data.
The goal of theoretical work in the area of external memory (EM) algorithms (also called I/O algorithms
or out-of-core algorithms) is to eliminate or minimize the I/O bottleneck through better algorithm
design.
具体的算法地址
http://www.cs.duke.edu/TPIE
http://www.madalgo.au.dk/Trac-tpie
TPIE, a Transparent Parallel I/O Environment, designed to assist programmers
in writing high performance I/O-efficient programs for a variety of platforms.
这个算法对IO进行了抽象，实现了栈、B树等算法
效率的测试方法
http://algo2.iti.kit.edu/dementiev/stxxl/trunk/containers_2berkeley__db__benchmark_8cpp-example.html