SciMark 2.0 (http://math.nist.gov/scimark2/) is a Java benchmark for scientific and numerical computing. It measures several computational kernels and reports a composite score in approximate Mflops/s. This benchmark was developed at the US National Institute of Standards and Technology (NIST). Part of the benchmark can also be found in the Java Grande Forum Benchmark Suite (http://www.epcc.ed.ac.uk/javagrande/javag.html). This benchmark contains codes on FFT, SOR (Successive Over-Relaxation over a 2D grid), Monte-Carlo integration, Sparse matmult (Sparse matrix vector multiplications) and LU factorization. We have chosen this benchmark because the same benchmark is available both in Java and C, allowing us to compare the two languages. There are many other Java benchmarks available, see http://www.epcc.ed.ac.uk/javagrande/links.html.
Table 4 contains the result of this benchmark. The
compiler flag used with gcc is -O2 -funroll-loops, which is the
flag that comes with the makefile of this benchmark. The loop unrolling
does seem to improve the performance of the LU factorization and the
sparse matrix vector multiplications, although it had minimal effect
on our previous sparse matrix multiplication benchmark, and was
therefore not used for that. We used two settings of the
benchmark: SMALL and LARGE. When the SMALL setting is chosen, the
problems tend to be able to fit into the cache. The LARGE setting is
useful in measuring the capability of the memory subsystem since
the size of the benchmark at that setting is
designed to be much bigger than most low-level caches (
2MB).
On the Pentium system, for the SMALL setting, Java (IBM) is
about 30% slower (
) than C, while for larger problems
it is about 40% slower. The Java (Sun) again does not perform
as well as Java (IBM), and is on average 33% slower than
Java (IBM) on SMALL setting and 12% slower on the LARGE setting.
On the Sun Ultra 80, Java is about 170% slower than C on the SMALL setting and 103% slower on the LARGE setting!
On the IBM Power3, Java is about 2.19 times slower than C on the SMALL setting and 1.97 times slower on the LARGE setting! As in Section 3, we did run the SciMark2 on a PowerPC Silver platform as well. It was found that on the IBM PowerPC Silver processor, Java was about 42% slower than C on the SMALL setting and only 24% slower on the LARGE setting. Again we do not understand why the performance of Java is so poor on the Power3.
| Pentium II | ||||||
| SMALL | LARGE | |||||
| Applications | gcc | Java (IBM) | Java (Sun) | gcc | Java (IBM) | Java (Sun) |
| FFT | 39.93 | 24.70 | 14.97 | 5.80 | 6.75 | 5.42 |
| SOR | 79.49 | 74.22 | 60.15 | 50.29 | 38.09 | 33.85 |
| MonteCarlo | 10.69 | 4.17 | 3.69 | 10.65 | 4.16 | 3.69 |
| Sparse matmult | 41.80 | 28.42 | 20.63 | 19.05 | 15.64 | 13.33 |
| LU | 57.08 | 24.70 | 14.97 | 27.71 | 6.75 | 5.42 |
| Composite Score | 45.80 | 35.50 | 26.63 | 22.70 | 16.24 | 14.50 |
| Sun Ultra 80 | ||||
| SMALL | LARGE | |||
| Applications | C | Java | C | Java |
| FFT | 63.38 | 17.27 | 10.71 | 7.32 |
| SOR | 135.32 | 54.66 | 89.96 | 44.87 |
| MonteCarlo | 17.25 | 5.82 | 16.99 | 5.88 |
| Sparse matmult | 48.91 | 22.40 | 30.48 | 17.27 |
| LU | 94.03 | 17.27 | 52.78 | 7.32 |
| Composite Score | 71.78 | 25.71 | 40.18 | 19.78 |
| IBM Power3 | ||||
| SMALL | LARGE | |||
| Applications | C | Java | C | Java |
| FFT | 117.22 | 78.37 | 10.89 | 11.31 |
| SOR | 147.32 | 94.19 | 142.47 | 91.72 |
| MonteCarlo | 22.60 | 5.07 | 22.60 | 5.09 |
| Sparse matmult | 120.92 | 71.89 | 122.49 | 75.16 |
| LU | 341.33 | 78.37 | 244.20 | 11.31 |
| Composite Score | 149.88 | 68.32 | 108.53 | 55.17 |