SciMark 2.0 (http://math.nist.gov/scimark2/) is a Java benchmark for scientific and numerical computing. It measures several computational kernels and reports a composite score in approximate Mflops/s. This benchmark was developed at the US National Institute of Standards and Technology (NIST). Part of the benchmark can also be found in the Java Grande Forum Benchmark Suite (http://www.epcc.ed.ac.uk/javagrande/javag.html). This benchmark contains codes on FFT, SOR (Successive Over-Relaxation over a 2D grid), Monte-Carlo integration, Sparse matmult (Sparse matrix vector multiplications) and LU factorization. We have chosen this benchmark because the same benchmark is available both in Java and C, allowing us to compare the two languages. There are many other Java benchmarks available, see http://www.epcc.ed.ac.uk/javagrande/links.html.
Table 4 contains the result of this benchmark. The compiler flag used with gcc is -O2 -funroll-loops, which is the flag that comes with the makefile of this benchmark. The loop unrolling does seem to improve the performance of the LU factorization and the sparse matrix vector multiplications, although it had minimal effect on our previous sparse matrix multiplication benchmark, and was therefore not used for that. We used two settings of the benchmark: SMALL and LARGE. When the SMALL setting is chosen, the problems tend to be able to fit into the cache. The LARGE setting is useful in measuring the capability of the memory subsystem since the size of the benchmark at that setting is designed to be much bigger than most low-level caches ( 2MB).
On the Pentium system, for the SMALL setting, Java (IBM) is about 30% slower () than C, while for larger problems it is about 40% slower. The Java (Sun) again does not perform as well as Java (IBM), and is on average 33% slower than Java (IBM) on SMALL setting and 12% slower on the LARGE setting.
On the Sun Ultra 80, Java is about 170% slower than C on the SMALL setting and 103% slower on the LARGE setting!
On the IBM Power3, Java is about 2.19 times slower than C on the SMALL setting and 1.97 times slower on the LARGE setting! As in Section 3, we did run the SciMark2 on a PowerPC Silver platform as well. It was found that on the IBM PowerPC Silver processor, Java was about 42% slower than C on the SMALL setting and only 24% slower on the LARGE setting. Again we do not understand why the performance of Java is so poor on the Power3.
|Applications||gcc||Java (IBM)||Java (Sun)||gcc||Java (IBM)||Java (Sun)|
|Sun Ultra 80|