Effect of Standard Template Classes on Performance

Tests in order of increasing abstraction; see discussion below for detailed description.
Capital letter means wrapped in a class, d:doubles, p:pointer, a:reverse-interator adaptor.
See below table for more details on benchmarking and link to benchmark code.
test
number 		0
Fort 		1
dp 		2
Dp 		3
dP 		4
DP 		5
dpa 		6
Dpa 		7
dPa 		8
DPa 		9
dpA 		10
DpA 		11
dPA 		12
DPA 		mean
IBM (lifu) with xlC -O3
abs time
second 		2.28 6.80 11.53 8.38 14.00 8.39 14.08 13.60 17.19 13.68 17.19 16.76 21.43 11.30
additions
M/second 		21.93 7.35 4.34 5.97 3.57 5.96 3.55 3.68 2.91 3.65 2.91 2.98 2.33 4.33
test/test0 1.00 2.98 5.06 3.68 6.14 3.68 6.18 5.96 7.54 6.00 7.54 7.35 9.40 4.96
IBM (lifu) with KCC +K3
abs time
second 		2.25 4.52 4.54 5.23 5.37 5.21 5.24 6.01 6.06 6.01 6.11 6.75 7.50 5.27
additions
M/second 		22.22 11.06 11.01 9.56 9.31 9.60 9.54 8.32 8.25 8.32 8.18 7.41 6.67 9.49
test/test0 1.00 2.01 2.02 2.32 2.39 2.32 2.33 2.67 2.69 2.67 2.72 3.00 3.33 2.34
SGI PC (OSC) with KCC +K3
abs time
second 		2.22 2.24 2.23 2.23 2.25 2.24 2.23 2.24 2.24 2.23 2.23 2.24 2.24 2.24
additions
M/second 		22.42 22.32 22.42 22.42 22.22 22.32 22.42 22.32 22.32 22.42 22.42 22.32 22.32 22.36
test/test0 1.00 1.00 1.00 1.00 1.01 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
Origin2000 CC -O3
abs time
second 		0.27 0.29 0.28 0.52 0.53 0.54 0.52 0.54 0.52 0.55 0.54 0.51 0.52 0.46
additions
M/second 		185.19 172.41 178.57 96.15 94.34 92.59 96.15 92.59 96.15 90.91 92.59 98.04 96.15 109.50
test/test0 1.00 1.07 1.04 1.93 1.96 2.00 1.93 2.00 1.93 2.04 2.00 1.89 1.93 1.69
T3E (OSC) CC -O3
abs time
second 		1.27 4.20 5.11 3.39 4.95 2.96 4.45 4.96 7.59 5.43 7.75 9.25 10.47 4.89
additions
M/second 		39.36 11.92 9.78 14.76 10.11 16.92 11.22 10.08 6.59 9.21 6.45 5.40 4.78 10.22
test/test0 1.00 3.30 4.03 2.67 3.89 2.33 3.51 3.91 5.97 4.27 6.10 7.28 8.24 3.85
T3E (OSC) KCC +K3 -O3
abs time
second 		1.26 0.88 0.90 0.95 0.93 0.88 0.88 0.94 0.93 0.89 0.93 0.92 0.93 0.94
additions
M/second 		39.70 57.06 55.34 52.41 53.77 56.77 56.50 53.25 53.71 56.32 53.73 54.50 53.75 53.40
test/test0 1.00 0.70 0.72 0.76 0.74 0.70 0.70 0.75 0.74 0.70 0.74 0.73 0.74 0.74
test
number 		0
Fort 		1
dP 		2
Dp 		3
dP 		4
DP 		5
dpa 		6
Dpa 		7
dPa 		8
DPa 		9
dpA 		10
DpA 		11
dPA 		12
DPA 		mean

Description of Tests

To verify how efficiently C++ (and in particular STL) is compiled by the present day compilers, benchmark outputs 13 numbers computed with increasing abstrctions. In the ideal world these numbers should be the same. In the real world, however, ...

The final number printed by the benchmark is a geometric mean of the performance degradation factors of individual tests. It claims to represent the factor the compiler penalizes the code for using C++ data abstraction features. Call this number "Abstraction Penalty."

As with any benchmark it is hard to prove such a claim; some people claim this test does not represent typical C++ usage. It is, however, a noteworthy fact that majority of the people who so object are responsible for C++ compilers with disproportionatly large Abstraction Penalty.

The structure of the benchmark is really quite simple. It adds 2000 doubles in an array 25000 times. It does it in 13 different ways that introduce more and more abstract ways of doing it.

0 - uses simple Fortran-like for loop.
1 - 12 use STL style accumulate template function with plus function object.
1, 3, 5, 7 ,9, 11 use doubles.
2, 4, 6, 8, 10, 12 use Double - double wrapped in a class.
1, 2 - use regular pointers.
3, 4 - use pointers wrapped in a class.
5, 6 - use pointers wrapped in a reverse-iterator adaptor.
7, 8 - use wrapped pointers wrapped in a reverse-iterator adaptor.
9, 10 - use pointers wrapped in a reverse-iterator adaptor wrapped
   in a reverse-iterator adaptor.
11, 12 - use wrapped pointers wrapped in a reverse-iterator adaptor
   wrapped in a reverse-iterator adaptor.
Your comments and suggestions are appreciated.
[Previous] [Wilkins Home Page]
[OSU Physics] [OSU Chemical Physics] [Center for Materials Research]
[College of Mathematical & Physical Sciences] [Ohio State University]
To cite this page:
Effect of Standard Template Classes on Performance
<http://www.physics.ohio-state.edu/~wilkins/computing/benchmark/STC.html>
[Thursday, 04-Dec-2008 17:14:06 EST]
Edited by: wilkins@pismo.mps.ohio-state.edu on Wednesday, 24-Jun-1998 16:36:03 EDT