原创 2004年07月14日 13:32:00


Nine Language Performance Round-up: Benchmarking Math & File I/O

Type: Dell Latitude C640 Notebook
CPU: Pentium 4-M 2GHz
RAM: 768MB
Hard Disk: IBM Travelstar 20GB/4500RPM
Video: Radeon Mobility 7500/32MB
OS: Windows XP Pro SP 1
File System: NTFS Here's what happens in each part of the benchmark:

32-bit integer math: using a 32-bit integer loop counter and 32-bit integer operands, alternate among the four arithmetic functions while working through a loop from one to one billion. That is, calculate the following (while discarding any remainders):

1 - 1 + 2 * 3 / 4 - 5 + 6 * 7 / 8 - ... - 999,999,997 + 999,999,998 * 999,999,999 / 1,000,000,000

64-bit integer math: same algorithm as above, but use a 64-bit integer loop counter and operands. Start at ten billion and end at eleven billion so the compiler doesn't knock the data types down to 32-bit.
64-bit floating point math: same as for 64-bit integer math, but use a 64-bit floating point loop counter and operands. Don't discard remainders.
64-bit floating point trigonometry: using a 64-bit floating point loop counter, calculate sine, cosine, tangent, logarithm (base 10) and square root of all values from one to ten million. I chose 64-bit values for all languages because some languages required them, but if a compiler was able to convert the values to 32 bits, I let it go ahead and perform that optimization.
I/O: Write one million 80-character lines to a text file, then read the lines back into memory.


Here are the benchmark results presented in both table and graph form. The Python and Python/Psyco results are excluded from the graph since the large numbers throw off the graph's scale and render the other results illegible. All scores are given in seconds; lower is better.

Time int


Visual C++ 9.6 18.8 6.4 3.5 10.5 48.8
Visual C# 9.7 23.9 17.7 4.1 9.9 65.3
gcc C 9.8 28.8 9.5 14.9 10.0 73.0
Visual Basic 9.8 23.7 17.7 4.1 30.7 85.9
Visual J# 9.6 23.9 17.5 4.2 35.1 90.4
Java 1.3.1 14.5 29.6 19.0 22.1 12.3 97.6
Java 1.4.2 9.3 20.2 6.5 57.1 10.1 103.1
Python/Psyco 29.7 615.4 100.4 13.1 10.5 769.1
Python 322.4 891.9 405.7 47.1 11.9 1679.0


Let's review the results by returning to the five questions that motivated these benchmarks. First, Java (at least, in the 1.4.2 version) performed very well on most benchmark components when compared to the .NET 2003 languages. If we exclude the trigonometry component, Java performed virtually identically to Visual C++, the fastest of Microsoft's languages. Unfortunately, the trigonometry performance of Java 1.4.2 can only be described as dismal. It was bafflingly bad--worse even than fully interpreted Python! This was especially puzzling given the much faster trigonometry performance of Java 1.3.1, and suggests that there may be more efficient ways to code the benchmark in Java. Perhaps someone with more experience with 1.4.2 can suggest a higher-speed workaround.

Java performed especially well (when discounting the strange trigonometry performance) compared to Microsoft's syntactically equivalent Visual J#. This discrepancy may be due to the additional overhead of the CLR engine (as compared to the overhead of the JVM), or may have something to do with Visual J# implementing only version 1.1.4 of the Java spec.

Second, Microsoft's claim that all four .NET 2003 languages compile into identical MSIL code seemed mostly true for the math routines. The integer math component produced virtually identical scores in all four languages. The long math, double math, and trig scores were identical in Visual C#, Visual Basic, and Visual J#, but the C++ compiler somehow produced impressively faster code for these benchmark components. Perhaps C++ is able to make better use of the Pentium 4's SSE2 SIMD extensions for arithmetic and trigonometry, but this is pure speculation on my part. The I/O scores fell into two clusters, with Visual Basic and Visual J# apparently using much less efficient I/O routines than Visual C# or Visual C++. This is a clear case where functionally identical source code does not compile into identical MSIL code.

Third, Java 1.4.2 performed as well as or better than the fully compiled gcc C benchmark, after discounting the odd trigonometry performance. I found this to be the most surprising result of these tests, since it only seems logical that running bytecode within a JVM would introduce some sort of performance penalty relative to native machine code. But for reasons unclear to me, this seems not to be true for these tests.

Fourth, fully interpreted Python was, as expected, much slower than any of the fully compiled or semi-compiled languages--sometimes by a factor of over 60. It should be noted that Python's I/O performance was in the same league as the fastest languages in this group, and was faster than Visual Basic and Visual J#. The Psyco compiler worked wonders with Python, reducing the time required for the math and trig components to between 10% and 70% of that required for Python without Psyco. This was an astonishing increase, especially considering how easy it is to include Psyco in a Python project.

Fifth, Java 1.4.2 was much faster than Java 1.3.1 in the arithmetic components, but as already mentioned, it lagged way behind the older version on the trigonometry component. Again, I can't help but think that there may be a different, more efficient way to call trigonometric functions in 1.4.2. Another possibility is that 1.4.2 may be trading accuracy for speed relative to 1.3.1, with new routines that are slower but more correct.

What lessons can we take away from all of this? I was surprised to see the four .NET 2003 languages clustered so closely on many of the benchmark components, and I was astonished to see how well Java 1.4.2 did (discounting the trigonometry score). It would be foolish to offer blanket recommendations about which languages to use in which situations, but it seems clear that performance is no longer a compelling reason to choose C over Java (or perhaps even over Visual J#, Visual C#, or Visual Basic)--especially given the extreme advantages in readability, maintainability, and speed of development that those languages have over C. Even if C did still enjoy its traditional performance advantage, there are very few cases (I'm hard pressed to come up with a single example from my work) where performance should be the sole criterion when picking a programming language. I would even argue that that for very complex systems that are designed to be in use for many years, maintainability ought to trump all other considerations (but that's an issue to take up in another article).

Expanding the Benchmark

The most obvious way to make this benchmark more useful is to expand it beyond basic arithmetic, trigonometry, and file I/O. I could also extend the range of languages or variants tested. For example, testing Visual Basic 6 (the last of the pre-.NET versions of VB) would give us an idea how much (if any) of a performance hit the CLR adds to VB. There are other JVMs available to be tested, including the open-source Kaffe and the JVM included with IBM's SDK (which seems to be stuck at version 1.3 of the Java spec). BEA has an interesting JVM called JRockit which promises performance improvements in certain situations, but unfortunately only works on Windows. GNU's gcj front-end to gcc allows Java source code to be compiled all the way to executable machine code, but I don't know how compatible or complete the package is. There are a number of other C compilers available that could be tested (including the highly regarded Intel C compiler), as well as a host of other popular interpreted languages like Perl, PHP, or Ruby. So there's plenty of room for further investigation.

I am by no means an expert in benchmarking; I launched this project largely as a learning experience and welcome suggestions on how to improve these benchmarks. Just remember the limited ambitions of my tests: I am not trying to test all aspects of a system--just a small subset of the fundamental operations on which all programs are built.

About the author:
Christopher W. Cowell-Shah works in Palo Alto as a consultant for the Accenture Technology Labs (the research & development wing of Accenture). He has an A.B. in computer science from Harvard and a Ph.D. in philosophy from Berkeley. Chris is especially interested in issues in artificial intelligence, human/computer interaction and security. His website is www.cowell-shah.com.


一.UML简介        Unified Modeling Language (UML)又称统一建模语言或标准建模语言。   UML有3种基本的构造块:1.事物、2.关系和3.图。       ...
  • z15732621736
  • z15732621736
  • 2015年10月04日 19:56
  • 1007


图解九种主流编程语言,从Python、Java到Objective-C。码农世界,哪种才是最好的编程语言? 这9张信息图囊括了九种主流编程语言:Python、Java、C、C++、JavaScr...
  • wuyusheng314
  • wuyusheng314
  • 2017年03月20日 16:21
  • 25360


一、Benchmark简介 Benchmark是一个评价方式,在整个计算机领域有着长期的应用。正如维基百科上的解释“As computer architecture advanced, it beca...
  • u012050154
  • u012050154
  • 2016年02月24日 15:12
  • 5138


原文链接1:http://wiki.jikexueyuan.com/project/node-lessons/benchmark.html 原文链接2:https://github.com/also...
  • hellochenlu
  • hellochenlu
  • 2015年11月28日 21:52
  • 975

Apache Benchmark安装、参数含义&使用总结、结果分析

首先,介绍下背景,我使用的系统是CentOS7.1。   Apache Benchmark简称AB,安装有两种方式: 1.使用sudo yum install httpd-tools 命令安装(比较简...
  • sangyongjia
  • sangyongjia
  • 2015年10月13日 11:02
  • 9517

使用Apache Benchmark做压力测试遇上的5个常见问题

文章转载自:http://mo2g.com/view/50/ 这一篇文章主要记录我在使用Apache Benchmark(一下检测ab)做网站压力测试的过程中,遇到的一些问题以及解决...
  • sangyongjia
  • sangyongjia
  • 2015年10月13日 10:25
  • 1434


java提供了一组基本数据类型,包括 boolean, byte, char, short,  int, long, float, double, void.  同时,java也提供了这些类型的封...
  • syc434432458
  • syc434432458
  • 2015年11月21日 14:04
  • 8465

关于 Benchmark 的几个思考

之前搭建的push layer 需要有一个性能指标,leader 让我做的这个性能测试.其实Go的工具GOTOOL里面是有go test 的,可以测benchmark ,并且是支持Parallel的,...
  • arthur_killer
  • arthur_killer
  • 2016年01月29日 22:06
  • 1156


java提供的九种基本数据类型:boolean、byte、char、short、int、long、float、double、void 以及它们的封装类:Boolean、Byte、Character、...
  • syf1970
  • syf1970
  • 2016年09月19日 22:02
  • 580

学习笔记:MySQL基准测试之Benchmark Suite

MySQL Benchmark Suite是除了“mysqlslap”命令行工具(http://blog.csdn.net/sweeper_freedoman/article/details/7355...
  • sweeper_freedoman
  • sweeper_freedoman
  • 2017年06月22日 20:38
  • 1891